Continual learning of a time series model using a mixture of HMMs with application to the IoT fuel sensor verification

• Autorzy: Głomb Przemysław , Cholewa Michał , Foszner Paweł , Bularz Jakub

Identyfikatory

DOI

10.15439/2023F1856

• Języki publikacji: EN

Abstrakty

EN

This paper presents an application of a mixture of Hidden Markov Models (HMMs) as a tool for verification of IoT fuel sensors. The IoT fuel sensors report the level of fuel in tanks of a petrol station, and are a key component for monitoring system reliability (billing), safety (fuel/oil leak detection) and security (theft prevention). We propose an algorithm for learning a mixture of HMMs based on a continual learning principle, i.e. it adapts the model while monitoring a sensor over time, signalling unexpected or anomalous sensor reports. We have tested the proposed approach on a real-life data of 15 fuel tanks being monitored with the FuelPrime system, where it has shown a very good performance (average area under ROC curve of 0.94) of detecting anomalies in the sensor data. Additionally we show that the proposed method can be used for trend monitoring and present qualitative analysis of the short and long term learning performance. The proposed method has promising performance score, the resulting model has a high degree of explainability, limited memory and computation requirements and can be easily generalized to other domains of sensor verification.

Słowa kluczowe

EN

computational modeling; time series analysis; hidden Markov models; sensor system; sensor; safety; security

PL

modelowanie obliczeniowe; analiza szeregów czasowych; ukryte modele Markowa; system czujników; czujnik; bezpieczeństwo

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2023
• Tom: Vol. 35
• Strony: 259--264
• Opis fizyczny: Bibliogr. 26 poz., il., wykr.

Twórcy

autor

Głomb Przemysław

• Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Bałtycka 5, 44-100 Gliwice, Poland

autor

Cholewa Michał

• Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Bałtycka 5, 44-100 Gliwice, Poland

autor

Foszner Paweł

• Department of Computer Graphics, Vision and Digital Systems Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology Akademicka 2A, 44-100 Gliwice, Poland

autor

Bularz Jakub

• AIUT Sp. z o.o. ul. Wyczółkowskiego 113, 44-109 Gliwice, Poland

Bibliografia

• 1. R. Krishnamurthi, A. Kumar, D. Gopinathan, A. Nayyar, and B. Qureshi, “An overview of IoT sensor data processing, fusion, and analysis techniques,” Sensors, vol. 20, no. 21, 2020. http://dx.doi.org/10.3390/s20216076
• 2. M. Henry and G. Wood, “Sensor validation: principles and standards,” atp International, vol. 3, no. 2, 2005.
• 3. A. Blázquez-García, A. Conde, U. Mori, and J. A. Lozano, “A review on outlier/anomaly detection in time series data,” ACM Computing Surveys, vol. 54, no. 3, 2021. http://dx.doi.org/10.1145/3444690
• 4. M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: Defying forgetting in classification tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366–3385, 2022. http://dx.doi.org/10.1109/TPAMI.2021.3057446
• 5. J. M. Ramírez, F. Díez, P. Rojo, V. Mancuso, and A. Fernández-Anta, “Explainable machine learning for performance anomaly detection and classification in mobile networks,” Computer Communications, vol. 200, pp. 113–131, 2023. http://dx.doi.org/10.1016/j.comcom.2023.01.003
• 6. N. Aslam, I. U. Khan, A. Alansari, M. Alrammah, A. Alghwairy, R. Alqahtani, R. Alqahtani, M. Almushikes, and M. A. Hashim, “Anomaly detection using explainable random forest for the prediction of undesirable events in oil wells,” Applied Computational Intelligence and Soft Computing, vol. 2022, p. 1558381, 2022. http://dx.doi.org/10.1155/2022/1558381
• 7. F. Dama and C. Sinoquet, Time Series Analysis and Modeling to Forecast: a Survey. https://arxiv.org/abs/2104.00164 [cs.LG], 2021.
• 8. M. Staniszewski, A. Skorupa, Ł. Boguszewicz, M. Sokół, and A. Polański, “Quality control procedure based on partitioning of NMR time series,” Sensors, vol. 18, no. 3, p. 792, 2018. http://dx.doi.org/10.3390/s18030792
• 9. M. Staniszewski, A. Skorupa, Ł. Boguszewicz, A. Michalczuk, K. Wereszczyński, M. Wicher, M. Konopka, M. Sokół, and A. Polański, “Application of reiteration of hankel singular value decomposition in quality control,” AIP Conference Proceedings, vol. 1863, no. 1, p. 400006, 2017. http://dx.doi.org/10.1063/1.4992575
• 10. L. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, 1989. http://dx.doi.org/10.1109/5.18626
• 11. E. Lach, D. Grzechca, A. Polański, J. Rutkowski, and M. Staniszewski, “Analysis of semestral progress in higher technical education with HMM models,” in Computational Science – ICCS 2021. Springer, 2021. http://dx.doi.org/10.1007/978-3-030-77967-2_18 pp. 214–228.
• 12. M. Cholewa and P. Głomb, “Natural human gestures classification using multisensor data,” in Proceedings 3rd IAPR Asian Conference on Pattern Recognition ACPR 2015. IEEE, 2015. http://dx.doi.org/10.1109/ACPR.2015.7486553 pp. 499–503.
• 13. J. Ying, T. Kirubarajan, K. Pattipati, and A. Patterson-Hine, “A hidden Markov model-based algorithm for fault diagnosis with partial and imperfect tests,” IEEE Transactions on Systems, Man, and Cybernetics, Part C, vol. 30, no. 4, pp. 463–473, 2000. http://dx.doi.org/10.1109/5326.897073
• 14. W. Zhao, T. Shi, and L. Wang, “Fault diagnosis and prognosis of bearing based on Hidden Markov Model with multi-features,” Applied Mathematics and Nonlinear Sciences, vol. 5, no. 1, pp. 71–84, 2020. http://dx.doi.org/10.2478/amns.2020.1.00008
• 15. F. van de Pol and R. Langeheine, “Mixed Markov latent class models,” Sociological Methodology, vol. 20, pp. 213–247, 1990. http://dx.doi.org/10.2307/271087
• 16. M. D. R. de Chaumaray, M. Marbac, and F. Navarro, “Mixture of hidden Markov models for accelerometer data,” Annals of Applied Statistics, vol. 14, no. 4, pp. 1834 – 1855, 2020. http://dx.doi.org/10.48550/arXiv.1906.01547
• 17. D. Vidotto, J. K. Vermunt, and K. V. Deun, “Multiple imputation of longitudinal categorical data through bayesian mixture latent Markov models,” Journal of Applied Statistics, vol. 47, no. 10, pp. 1720–1738, 2020. http://dx.doi.org/10.1080/02664763.2019.1692794
• 18. W. Khreich, E. Granger, A. Miri, and R. Sabourin, “Adaptive ROC-based ensembles of HMMs applied to anomaly detection,” Pattern Recognition, vol. 45, no. 1, pp. 208–230, 2012. http://dx.doi.org/10.1016/j.patcog.2011.06.014
• 19. P. R. Cavalin, R. Sabourin, C. Y. Suen, and A. S. Britto Jr., “Evaluation of incremental learning algorithms for HMM in the recognition of alphanumeric characters,” Pattern Recognition, vol. 42, no. 12, pp. 3241–3253, 2009. http://dx.doi.org/10.1016/j.patcog.2008.10.012
• 20. W. Khreich, E. Granger, A. Miri, and R. Sabourin, “A survey of techniques for incremental learning of HMM parameters,” Information Sciences, vol. 197, pp. 105–130, 2012. http://dx.doi.org/10.1016/j.ins.2012.02.017
• 21. K. Faber, R. Corizzo, B. Sniezynski, and N. Japkowicz, “Vlad: Task-agnostic vae-based lifelong anomaly detection,” Neural Networks, vol. 165, pp. 248–273, 2023. http://dx.doi.org/10.1016/j.neunet.2023.05.032
• 22. D. Samariya and A. Thakkar, “A comprehensive survey of anomaly detection algorithms,” Annals of Data Science, vol. 10, no. 3, pp. 829–850, 2023. http://dx.doi.org/10.1007/s40745-021-00362-9
• 23. L. Wang, X. Zhang, H. Su, and J. Zhu, A Comprehensive Survey of Continual Learning: Theory, Method and Application. https://arxiv.org/abs/2104.00164 [cs.LG], 2023.
• 24. P. Głomb, M. Romaszewski, A. Sochan, and S. Opozda, “Unsupervised parameter selection for gesture recognition with vector quantization and hidden markov models,” in Human-Computer Interaction – INTERACT 2011, P. Campos, N. Graham, J. Jorge, N. Nunes, P. Palanque, and M. Winckler, Eds. Springer Berlin Heidelberg, 2011. http://dx.doi.org/10.1007/978-3-642-23768-3_14 pp. 170–177.
• 25. P. Foszner, A. Gruca, and J. Bularz, “Fuel pipeline thermal conductivity in automatic wet stock reconciliation systems,” in Advances in Data Mining. Applications and Theoretical Aspects: 16th Industrial Conference. Springer, 2016. http://dx.doi.org/10.1007/978-3-319-41561-1_22 pp. 297–310.
• 26. G. Schwarz, “Estimating the dimension of a model,” Annals of Statistics, vol. 6, pp. 461–464, 1978. http://dx.doi.org/10.1214/aos/1176344136

Uwagi

1. This work was partially funded by National Center for Research and Development no POIR.01.01.01-00-0376/17-00 ‘A system for gathering and analysis of streaming data for fuel stations, for optimization of distribution costs and fuel sales as well as on-line monitoring for leakages-related issues’ (FuelPrime).

2. Main Track Short Papers

3. Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).