Emotion recognition from heart rate variability using a hybrid system combined with a Hidden Markov model and Poincare plot

• Autorzy: Khanghah Sahar Zamani , Maghooli Keivan

Identyfikatory

DOI

10.35784/acs-2024-07

• Języki publikacji: EN

Abstrakty

EN

The best emotion recognition system based on physiological signals with a simple operatory should have higher accuracy and fast response speed. This paper aims to develop an emotion recognition system using a novel hybrid system based on Hidden Markov Model and Poincare plot. For this purpose, an electrocardiogram from the MAHNOB-HCI database was used. A novel feature extraction from a hybrid system combining Hidden Markov Model and Poincare plot was presented. The authors extracted time and frequency domain features from heart rate variability, and used two central hybrid systems, the Support Vector Machine/ Hidden Markov Model and the Hidden Markov Model/ Poincare Plot. Finally, the support vector machine was used as a classifier to classify emotions into positive and negative. The proposed method showed a classification accuracy of 95.02 ± 1.97 % overall. Also, the computing time of the method is around 163 milliseconds. The key of this paper is in the use of hybrid machines to improve accuracy without high computation time. This method can be used as a real-time system due to the low computation time and can be developed in many fields, such as medical examination and security systems.

Słowa kluczowe

EN

electrocardiogram; features extraction; support vector machine; emotion classification

• Wydawca: Polskie Towarzystwo Promocji Wiedzy ; Lublin University of Technology
• Czasopismo: Applied Computer Science
• Rocznik: 2024
• Tom: Vol. 20, no 1
• Strony: 106--121
• Opis fizyczny: Bibliogr. 26 poz., fig., tab.

Twórcy

autor

Khanghah Sahar Zamani

• Islamic Azad University, Department of Biomedical Engineering, Central Tehran Branch, Tehran, Iran

• https://orcid.org/0009-0009-0485-092X

autor

Maghooli Keivan

• Islamic Azad University, Department of Biomedical Engineering, Science and Research Branch, Tehran, Iran

• https://orcid.org/0000-0003-0980-0154

Bibliografia

• [1] Akbulut, F. P., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Computer Methods and Programs in Biomedicine, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
• [2] Arias-Londoño, J. D., & Godino-Llorente, J. I. (2015). Entropies from Markov models as complexity measures of embedded attractors. Entropy, 17(6), 3595-3620. https://doi.org/10.3390/e17063595
• [3] Baghizadeh, M., Maghooli, K., Farokhi, F., & Dabanloo, N. J. (2020). A new emotion detection algorithm using extracted features of the different time-series generated from ST intervals Poincaré map. Biomedical Signal Processing and Control, 59, 101902. https://doi.org/10.1016/j.bspc.2020.101902
• [4] Bong, S. Z., Murugappan, M., & Yaacob, S. (2012). Analysis of electrocardiogram (ECG) signals for human emotional stress classification. International Conference on Intelligent Robotics, Automation, and Manufacturing (IRAM 2012) (pp. 198-205). Springer. https://doi.org/10.1007/978-3-642-35197-6_22
• [5] Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20, 100363. https://doi.org/10.1016/j.imu.2020.100363
• [6] Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63, 024001. https://doi.org/10.1088/1361-6587/abcbaa
• [7] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273-297. https://doi.org/10.1007/BF00994018
• [8] Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) (pp. 1-6). IEEE. https://doi.org/10.1109/CIBCB.2016.7758108
• [9] Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australasian Physical & Engineering Sciences in Medicine, 40, 277-287. https://doi.org/10.1007/s13246-017-0530-x
• [10] Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomedical Engineering / Biomedizinische Technik, 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
• [11] Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Autonomic Neuroscience, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
• [12] Tawsif, K., Nor Azlina, A. A., Emerson, R. J., Hossen, J., & Jesmeen, M. Z. H. (2022). A Systematic review on emotion recognition system using physiological signals: Data acquisition and methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
• [13] Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
• [14] Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42, 419-427. https://doi.org/10.1007/BF02344719
• [15] Krüger, S. E., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. Proc. Interspeech 2005, 993-996. https://doi.org/10.21437/Interspeech.2005-237
• [16] Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A self-adaptive hidden markov model for emotion classification in chinese microblogs. Mathematical Problems in Engineering, 2015, 987189. https://doi.org/10.1155/2015/987189
• [17] Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
• [18] Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion recognition in human computer interaction systems. Elektronika ir Elektrotechnika, 20(10), 51-56. https://doi.org/10.5755/j01.eee.20.10.8878
• [19] Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
• [20] Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Archives of Psychiatric Nursing, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
• [21] Shaffer, F., & Ginsberg, J. P. (2017). An overview of heart rate variability metrics and norms. Frontiers in Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
• [22] Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
• [23] Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition. 8th International Conference on Spoken Language Processing (INTERSPEECH 2004 - ICSLP) (pp. 661-664). https://doi.org/10.21437/Interspeech.2004-265
• [24] Wang, B., Liu, D., Gao, X., & Luo, Y. (2022). Three-dimensional poincaré plot analysis for heart rate variability. Complexity, 2022, 3880047. https://doi.org/10.1155/2022/3880047
• [25] Wiem, M., & Lachiri, Z. (2017). Emotion classification in arousal valence model using MAHNOB-HCI database. International Journal of Advanced Computer Science and Applications, 8(3), 318-323. https://doi.org/10.14569/IJACSA.2017.080344
• [26] Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiological Measurement, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887