Sign language classifier based on machine learning

• Autorzy: Avram Camelia , Păcurar Laura-Flavia , Radu Dan

Identyfikatory

DOI

10.7862/tiam.2024.1.2

Warianty tytułu

PL

Klasyfikator języka migowego oparty na uczeniu maszynowym

• Języki publikacji: EN

Abstrakty

EN

Sign language represents an efficient way for individuals with hearing impairments to communicate. We propose a sign recognition system into which several tools are integrated to help with the image pre-processing part. By doing so, a machine learning model was developed that does not require a lot of processing power because instead of using the images themselves, it uses extracted data from them to connect this model to a mobile interface that the users will use to recognise signed letters successfully. The communication between the client and the model is sustained through a local server. Introducing sign language into assembly processes is not only a gesture of respect for diversity and inclusion but also a strategic decision that brings tangible benefits. It improves communication, safety, employee morale and overall efficiency, an essential element in achieving operational excellence and an integrated workplace.

PL

Język migowy stanowi skuteczny sposób komunikacji dla osób z upośledzeniem słuchu. Proponujemy system rozpoznawania znaków, w którym zintegrowano kilka narzędzi pomagających we wstępnym przetwarzaniu obrazu. W ten sposób opracowano model uczenia maszynowego, który nie wymaga dużej mocy obliczeniowej, ponieważ zamiast korzystać z samych obrazów, wykorzystuje wyodrębnione z nich dane, aby połączyć ten model z interfejsem mobilnym, którego użytkownicy będą używać do skutecznego rozpoznawania podpisanych liter. Komunikacja między klientem a modelem odbywa się za pośrednictwem lokalnego serwera. Wprowadzenie języka migowego do procesów montażowych to nie tylko gest szacunku dla różnorodności i integracji, ale także strategiczna decyzja, która przynosi wymierne korzyści. Poprawia komunikację, bezpieczeństwo, morale pracowników i ogólną wydajność, co jest istotnym elementem w osiąganiu doskonałości operacyjnej i zintegrowanego miejsca pracy.

Słowa kluczowe

EN

sign language in assembly process; sign language classifier; machine learning; mobile application

PL

język migowy w procesie montażu; klasyfikator języka migowego; uczenie maszynowe; aplikacja mobilna

• Wydawca: Oficyna Wydawnicza Politechniki Rzeszowskiej ; Sieć Badawcza Łukasiewicz - Warszawski Instytut Technologiczny
• Czasopismo: Technologia i Automatyzacja Montażu
• Rocznik: 2024
• Tom: nr 1
• Strony: 10--15
• Opis fizyczny: Bibliogr. 23 poz., il. kolor.

Twórcy

autor

Avram Camelia

• Department of Automation, Technical University of Cluj Napoca, Ge. Bariţiu 26-28, Cluj-Napoca, Romania

• https://orcid.org/0000-0002-7394-5051

autor

Păcurar Laura-Flavia

• Department of Automation, Technical University of Cluj Napoca, Ge. Bariţiu 26-28, Cluj-Napoca, Romania

autor

Radu Dan

• Department of Automation, Technical University of Cluj Napoca, Ge. Bariţiu 26-28, Cluj-Napoca, Romania

Bibliografia

• Adithya V. and Rajesh R.. 2020. A deep convolutional neural network approach for static hand gesture recognition. Procedia Comput. Sci. 171 (2020), 2353-2361.
• ALTA (2024). Deaf culture and sign language. https://www.altalang.com/beyond-words/deaf-culture-and-sign-language/
• Assael, Y. M., Shillingford, B., Whiteson, S., & de Freitas, N. (2016). LipNet: Sentence-level Lipreading. Computing Research Repository (CoRR), 1611.01599, 1-13. http://arxiv.org/abs/1611.01599
• Bhavsar, H. (2017). Review of Classification Methods Used in Image-based Sign Language Recognition System. International Journal on Recent and Innovation Trends in Computing and Communication, 5(5), 949-959.
• Bhavsar, H., & Trivedi, J. (2017). Review of feature extraction methods for image-based sign language recognition systems. Indian Journal of Computer Science and Engineering (IJCSE), 8(3), 249-259.
• Cao, Y., Chou, P. A., Zhang, Z., & Park, H. S. (2018). Recent advancements in deep learning for sign language recognition: A review. Computer Vision and Image Understanding, 171, 130-140.
• Chai, X., Li, G., Lin, Y., Xu, Z., Tang, Y., Chen, X., & Zhou, M. (2013). Sign language recognition and translation with Kinect. In Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition (pp. 1-2). Shanghai, China.
• Costa Filho, C. F. F., Souza, R. S. de, Santos, J. R. dos, Santos, B. L. dos, & Costa, M. G. F. (2017). A fully automatic method for recognising hand configurations of Brazilian sign language. Research on Biomedical Engineering, 33(1), 78-89.
• Earthweb (2024). Sign language users. https://earthweb. com/sign-language-users/.
• Elmezain, M., Al-Hamadi, A., Appenrodt, J., & Michaelis, B. (2008). A hidden Markov model-based continuous gesture recognition system for hand motion trajectory. In Pattern Recognition, 2008. ICPR 2008. 19th International Conference on (pp. 1-4). Tampa, FL, USA.
• Escobedo, E., & Camara, G. (2016). A new Approach for Dynamic Gesture Recognition using Skeleton Trajectory Representation and Histograms of Cumulative Magnitudes. In SIBGRAPI Conference on Graphics, Patterns, and Images (pp. 209-216). São Paulo, Brazil.
• Ghosh, D. K., & Ari, S. (2016). On an algorithm for Vision-based hand gesture recognition. Signal, Image and Video Processing, 10(4), 655-662.
• Jayanthi Palraj, R K Ponsy, K Swetha, S A Subash. (2022). Real-Time Static and Dynamic Sign Language Recognition using Deep Learning. Journal of Scientific & Industrial Research 81(11):1186-1194. DOI: 10.56042/jsir.v81i11. 52657.
• Kakde, M. U., Nakrani, M. G., & Rawate, A. M. (2016). A Review Paper on Sign Language Recognition System For Deaf And Dumb People using Image Processing. International. Journal of Engineering Research and Technology, 5(3), 590-592.
• Kalsh, E. A., & Garewal, N. S. (2013). Sign Language Recognition System. International Journal of Computational Engineering Research, 3(6), 15-21.
• Kaluri, R., & Pradeep, C. H. (2017). An enhanced framework for sign gesture recognition using hidden Markov model and adaptive histogram technique. International Journal of Intelligent Engineering & Systems,10(3), 11-19.
• Manakitsa N, Maraslidis GS, Moysis L, Fragulis GF. (2024). A Review of Machine Learning and Deep Learning for Object Detection, Semantic Segmentation, and Human Action Recognition in Machine and Robotic Vision. Technologies.12(2):15. https://doi.org/10.3390/technologies12020015.
• MediaPipe, “MediaPipe Documenation”, [Online]. Available: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker.
• Naik BT, Hashmi MF, Bokde ND. (2022). A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions. Applied Sciences. 2022; 12(9):4429. https://doi.org/10.3390/app12094429.
• Nimratveer Kaur Bahia, Rajneesh Rani. (2023). Multi-level Taxonomy Review for Sign Language Recognition: Emphasis on Indian Sign Language. ACM Transactions on Asian and Low-Resource Language Information ProcessingVolume 22Issue 1Article No.: 23pp 1-39https://doi.org/10.1145/3530259.
• Siddharth Rautaray, Anupam Agrawal, (2012). A comprehensive review of vision-based hand gesture recognition. Artificial Intelligence Review 43(1). DOI: 10.1007/s10462-012-9356-9.
• Suharjitom F. W. G. P. K., Meita Chandra Ariesta. (2018). A Survey of Hand Gesture Recognition Methods in Sign. Pertanika Journal of Science and Technology, p. 19, 2018.
• The Alliance for Access to Computing Careers (2024). Deaf or Hard of Hearing. https://www.washington.edu/accesscomputing/what-sign-language.