Searching form the model of human facial expressions and its implementation in MATLAB and Simulink Environments

• Autorzy: Tchórzewski Jerzy Rudolf , Rosiński Jakub

Identyfikatory

DOI

10.34739/si.2025.32.07

• Języki publikacji: EN

Abstrakty

EN

The publication contains selected research results on the creation of a neural model of human facial expressions along with its implementation in the MATLAB and Simulink environments using the library called Deep Learning Toolbox. The model was generated as an object to recognize selected human faces and emotional states based on visual data recorded on the human face in real time. The study was placed against the background of available literature on the analysis of facial expressions and emotion classification methods. In addition to the concept of the original solution, the assumptions of the research experiment were given, a method for measuring facial expressions was selected for the experimental conditions, and a set of data was developed for training a neural model of the facial expression system with their preparation for ANN learning. Ultimately, various artificial neural networks were trained to model the facial expression system and sensitivity and comparative tests were performed in Simulink to, among others: assess the quality of the model in relation to real data. Very high results of ANN training of the facial expression system model were obtained (MSE on the order of 10−14, R close to 1), as well as relatively high model quality relative the facial expression system, measured, among others, average relative error, the value of which was several percent. However, in terms of prediction effectiveness, because of the obtained results, which are not very high for the assumed high measurement accuracies, the research that has been started is continued.

Słowa kluczowe

EN

Artificial Neural Networks; cluster analysis; facial expression recognition; image analysis; MATLAB; Simulink

• Wydawca: Wydawnictwo Uniwersytetu Przyrodniczo-Humanistycznego w Siedlcach
• Czasopismo: Studia Informatica : systems and information technology
• Rocznik: 2025
• Tom: vol. 1(32)
• Strony: 93--111
• Opis fizyczny: Bibliogr. 39 poz., rys., tab., wykr.

Twórcy

autor

Tchórzewski Jerzy Rudolf

• University of Siedlce, Faculty of Exact and Natural Sciences, Institute of Computer Science, ul. 3 Maja 54, 08-110 Siedlce, Poland

• https://orcid.org/0000-0003-2198-7185

autor

Rosiński Jakub

• Computer Science Graduate (Engineer of Computer Science) University of Siedlce, Faculty of Exact and Natural Sciences, Institute of Computer Science, ul. 3 Maja 54, 08-110 Siedlce, Poland

• https://orcid.org/0009-0003-8686-2524

Bibliografia

• 1. M. Adamiak, Wykorzystanie technik uczenia maszynowego i teledetekcji do wspomagania analiz, Uniwersytet Łódzki. 2022. URL: https:www.bip.uni.lodz.pl/fileadmin/user_upload/Adamiak_M.__prof._S.Mikrut.pdf [access: 2024-25].
• 2. A. Bulat and G. Tzimiropoulos, „How far are we from solving the 2 D & 3 D face alignment problem ?”, Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017, pp. 1021–1030.
• 3. M. Białas and A. Kowalska, „Analiza ekspresji twarzy w czasie rzeczywistym z użyciem biblioteki Open CVi uczenia maszynowego”. Informatyka i Automatyka, Vol. 30.4(2021), pp. 45–53.
• 4. I. Cohen and others, „Facial expression recognition from video sequences: Temporal and static modeling”. Computer Vision and Image Understanding 91.1–2 (2003), pp. 160–187
• 5. T. F. Cootes, G. J. Edwards and C. J. Taylor, „Active appearance models”. IEEE Transactions on Pattern Analysis and Machine Intelligence 23.6 (2001), pp. 681–685.
• 6. P. Ekman, „Facial expressions of emotion: New findings, new questions”, Psychological Science, Jan. 1992, Vol. 3, No. 1, pp. 33-38.
• 7. P. Ekman, Telling Lies: Clues to deceit in the marketplace, politics, and marriage, W. W. Norton & Company, 2009.
• 8. Guide for MATLAB, Guide for Simulink, Guide for System Identification Toolbox, Guide for Control System Toolbox, Guide for Neural Network Toolbox, Guide for Deep Learning Toolbox. Getting Started Guide, versions 2019b–2024b. The MathWorks®.
• 9. Google Research, Media Pipe – biblioteka do przetwarzania sygnałów wizualnych w czasie rzeczywistym.https://mediapipe.dev/.. 2023 [access: 2024-25].
• 10. A. Horzyk, Sztuczne systemy skojarzeniowe i asocjacyjna sztuczna inteligencja, Exit, Warszawa 2013, pages 277.
• 11. H. Jia, Z. Xiao and P. Ji, "Fatigue Driving Detection Based on Deep Learning and Multi-Index Fusion", IEEE Access, Vol. 9, pp. 147054-147062, 2021.
• 12. M. Janik, „Rozpoznawanie emocji na podstawie analizy mimiki twarzy z wykorzystaniem głębokich sieci neuronowych”, Praca doktorska, Politechnika Warszawska, Wydział Elektroniki i Technik Informacyjnych, Warszawa 2020.
• 13. V. Kazemi and J. Sullivan, „One millisecond face alignment with an ensemble of regression trees”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. 2014, pp. 1867–1874.
• 14. B.J.Kowalski, „Zastosowanie algorytmów klasyfikacyjnych w rozpoznawaniu emocji na podstawie mimiki twarzy”, Informatyka i Automatyka Vol. 27.2(2018), pp. 89–96.
• 15. Z. Liu, Y. Peng, W. Hu, Driver fatigue detection based on deeply-learned facial expression representation, Journal of Visual Communication and Image Representation, Volume 71, August 2020.
• 16. S. Li and W. Deng, "Deep Facial Expression Recognition: A Survey," in IEEE Transactions on Affective Computing, Vol. 13, No. 3, pp. 1195-1215, July-Sept, 2022.
• 17. M. J. Lyons, J. Budynek and S. Akamatsu. „Automatic classification of facial expressions using Gabor wavelets”. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition. IEEE. 1999, pp. 200–205.
• 18. G.Littlewortandothers,„Dynamicsoffacialexpressionextractedautomaticallyfromvideo”,Image and Vision Computing, Vol. 24.6(2006), pp. 615–625.
• 19. M. Laszczak, „Umiejętność deszyfracji mimiki twarzy – studium przypadku”, TBSP UJ, Kraków 2015, https:tbsp.wpia.uj.edu.pl/documents/4137545/108606169/Laszczak_5%20_2015.pdf [access: 2024-2025].
• 20. MATLAB and Simulink Documentation. Online documentation for MATLAB R2022b-R2024b. The MathWorks®. 2023-2025.
• 21. „Mimikra emocje w kontekście społecznym. Znaczenie mimicznego naśladowania emocji”, SWPS Projekty Naukowe (2023), SWPS Uniwersytet Humanistyczno-Społeczny. https://swps.pl/nauka-i-badania/projekty/42-projekty-naukowe/20712-mimikraemocji-w-kontekscie-spolecznym, [acces: 2024-25].
• 22. A. Nowakowski, Emocje i ich rozpoznawanie na podstawie ekspresji mimicznej. Wydawnictwo Uniwersytetu Śląskiego, Katowice 2018.
• 23. S. Osowski, Sieci neuronowe do przetwarzania informacji. OW Politechniki Warszawskiej, Warszawa 2013, pages 422.
• 24. M. Pantic i L. J. M. Rothkrantz, „Automatic analysis of facial expressions: The state of the art", IEEE Transactions on Pattern Analysis and Machine Intelligence (2006).
• 25. T. Pfister and others, „Recognising spontaneous facial micro-expressions”, IEEE International Conference on Computer Vision (ICCV), IEEE, 2011, pp. 1449–1456.
• 26. Polsko-Japońska Akademia Technik Komputerowych. „Systemy wykrywania i rozpoznawania twarzy na obrazach”. Przegląd Policyjny 4(140)(2020). URL: https://pja.edu.pl/wpcontent/uploads/2023/02/Systemy-wykrywania-i-rozpoznawania-twarzy-na-obrazach.pdf [access: 2024-25].
• 27. B. Paradowski, Sztuczna inteligencja w analizie zachowań ludzkich, Warszawa, WN PWN, Warszawa 2019.
• 28. J. Rosiński, Model mimiki twarzy człowieka i jego implementacja w środowsku MATLABA i Simulinka, praca inżynierska napisana w Instytucie Informatyki na Wydziale Nauk Ścisłych i Przyrodniczych pod kierunkiem dr hab. inż. J. Tchórzewskiego, prof. uczelni, Uniwersytet w Siedlcach, Siedlce 2025, pages 51.
• 29. C.Shan, S.Gongand P.W. McOwan, „Facial expression recognition based on Local Binary Patterns: Acomprehensive study”, Image and Vision Computing, Vol. 27.6(2009), pp. 803–816.
• 30. J. Tchórzewski, Metody sztucznej inteligencji i informatyki kwantowej w ujęciu teorii sterowania i systemów, WN UPH, Siedlce 2021, pages 343.
• 31. J. Tchórzewski, A. Wielgo, Mobile humanoid robots and the possibility of neural modeling human motion, Chapter 3 [in:] Intelligent Systems and Information Technologies. Modeling and Analysis of Intelligent Information Systems [ed.] J. Tchórzewski and P. Świtalski, No. 2, University of Siedlce, Siedlce 2023, pp. 67-98.
• 32. D. Wojciechowski, „Integracja sieci neuronowych w analizie sekwencji mimiki twarzy w czasie rzeczywistym”, Techniki Obliczeniowe, Vol. 34.1(2020), pp. 45–53.
• 33. S. Wierzchoń and M. Kłopotek, Algorytmy analizy skupień. Warszawa, WNT, Warszawa 2015, pages 395.
• 34. M. Valstar and M. Pantic, „Fully automatic recognition of the temporal phases of facial actions”. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), Vol. 36.1(2006), pp. 61–75.
• 35. L. Yin and others, „A 3D facial expression database for facial behavior research”, 8th IEEE International Conference on Automatic Face & Gesture Recognition, IEEE, 2008, pp. 211–216.
• 36. X. Zhu and oters, „Face alignment across large poses: A 3D solution”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 146–155.
• 37. Y.Zhang,Q.Ji,S.Gong, „ Real-time facial expression recognition based on hierarchical probabilistic models”, Proceedings of the IEEEConferenceon Computer Vision and Pattern Recognition(CVPR), 2005, pp. 118–125.
• 38. A. Talukder, S. Ghosh, Facial Image expression recognition and prediction system. Sci Rep 14, 27760. 2024.
• 39. M. Fatchan, M. Hery Purnomo, Affandy, A. Zainul Fanani and L. Marlinda, "Support Vector Machine and Neural Network Algorithm Approach to Classifying Facial Expression Recognition," 2020 Fifth International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesia, 2020, pp. 1-5.