Recognition of Sign Language from High Resolution Images Using Adaptive Feature Extraction and Classification

• Autorzy: Csóka Filip , Polec Jaroslav , Csóka Tibor , Kačur Juraj

Identyfikatory

DOI

10.24425/ijet.2019.126314

• Języki publikacji: EN

Abstrakty

EN

A variety of algorithms allows gesture recognition in video sequences. Alleviating the need for interpreters is of interest to hearing impaired people, since it allows a great degree of self-sufficiency in communicating their intent to the non-sign language speakers without the need for interpreters. State-of-the-art in currently used algorithms in this domain is capable of either real-time recognition of sign language in low resolution videos or non-real-time recognition in high-resolution videos. This paper proposes a novel approach to real-time recognition of fingerspelling alphabet letters of American Sign Language (ASL) in ultra-high-resolution (UHD) video sequences. The proposed approach is based on adaptive Laplacian of Gaussian (LoG) filtering with local extrema detection using Features from Accelerated Segment Test (FAST) algorithm classified by a Convolutional Neural Network (CNN). The recognition rate of our algorithm was verified on real-life data.

Słowa kluczowe

EN

sign language; gesture; sign; recognition; CNN; LoG; real-time; pattern recognition; machine learning

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2019
• Tom: Vol. 65, No. 2
• Strony: 303--308
• Opis fizyczny: Bibliogr. 17 poz., wykr., fot., rys.

Twórcy

autor

Csóka Filip

• Institute of Multimedia Information and Communication Technologies - Faculty of Electrical Engineering and Information Technology STU in Bratislava, Slovakia

autor

Polec Jaroslav

• Institute of Multimedia Information and Communication Technologies - Faculty of Electrical Engineering and Information Technology STU in Bratislava, Slovakia

autor

Csóka Tibor

• Institute of Multimedia Information and Communication Technologies - Faculty of Electrical Engineering and Information Technology STU in Bratislava, Slovakia

autor

Kačur Juraj

• Institute of Multimedia Information and Communication Technologies - Faculty of Electrical Engineering and Information Technology STU in Bratislava, Slovakia

Bibliografia

• [1] H. Chen, T. Ballal, M. Saad and T. Y. Al-Naffouri, "Angle-of-arrival-based gesture recognition using ultrasonic multi-frequency signals," 2017 25th European Signal Processing Conference (EUSIPCO), Kos, 2017, pp. 16-20.
• [2] S. Routray, A. K. Ray and C. Mishra, "Analysis of various image feature extraction methods against noisy image: SIFT, SURF and HOG," 2017 Second International Conference on Electrical, Computer and Communication Technologies (ICECCT), Coimbatore, 2017, pp. 1-5.
• [3] Q. B. Dang, V. P. Le, M. M. Luqman, M. Coustaty, C. D. Tran and J. M. Ogier, "Camera-based document image retrieval system using local features - comparing SRIF with LLAH, SIFT, SURF and ORB," 2015 13th International Conference on Document Analysis and Recognition (ICDAR), Tunis, 2015, pp. 1211-1215.
• [4] K. O. Rodríguez and G. C. Chávez, "Finger Spelling Recognition from RGB-D Information Using Kernel Descriptor," 2013 XXVI Conference on Graphics, Patterns and Images, Arequipa, 2013, pp. 1-7.
• [5] A. Anand, S. S. Tripathy and R. S. Kumar, "An improved edge detection using morphological Laplacian of Gaussian operator," 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN), Noida, 2015, pp. 532-536.
• [6] A. Anand, S. S. Tripathy and R. S. Kumar, "An improved edge detection using morphological Laplacian of Gaussian operator," 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN), Noida, 2015, pp. 532-536.
• [7] H. Kong, H. C. Akakin and S. E. Sarma, "A Generalized Laplacian of Gaussian Filter for Blob Detection and Its Applications," in IEEE Transactions on Cybernetics, vol. 43, no. 6, pp. 1719-1733, Dec. 2013.
• [8] Y. Biadgie and K. A. Sohn, "Feature Detector Using Adaptive Accelerated Segment Test," 2014 International Conference on Information Science & Applications (ICISA), Seoul, 2014, pp. 1-4.
• [9] L. Guo, J. Li, Y. Zhu and Z. Tang, "A novel Features from Accelerated Segment Test algorithm based on LBP on image matching," 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi'an, 2011, pp. 355-358.
• [10] D. Ravenscroft et al., "Learning feature extractors for AMD classification in OCT using convolutional neural networks," 2017 25th European Signal Processing Conference (EUSIPCO), Kos, 2017, pp. 51-55.
• [11] H. Yang, C. Yuan, J. Xing and W. Hu, "SCNN: Sequential convolutional neural network for human action recognition in videos," 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 390-394.2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 355-359.
• [12] Q. Wang, H. Fan, Y. Cong and Y. Tang, "Large receptive field convolutional neural network for image super-resolution," 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 958-962.
• [13] C. C. Hsu and C. W. Lin, "Unsupervised convolutional neural networks for large-scale image clustering," 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 390-394.
• [14] Zhe Xu, Biao Min, Ray C.C. Cheung, “Key-point Detection based Fast CU Decision for HEVC Intra Encoding” in International Journal of Electronics and Telecommunications, 2018, Volume 64, No. 3, ISSN:2081-8491
• [15] Sebastian Temich, Damian Grzechca, “Application of Neural Network for Testing selected specification parameters of Voltage-Controlled Oscillator” in International Journal of Electronics and Telecommunications, 2018, Volume 64, No. 2, ISSN:2081-8491.
• [16] Y. Zhao and L. Wang, "The Application of Convolution Neural Networks in Sign Language Recognition," 2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP), Wanzhou, China, 2018, pp. 269-272.
• [17] W. Fakhr, M. Kamel and M. I. Elmastry, "Probability of error, maximum mutual information, and size minimization of neural networks," [Proceedings 1992] IJCNN International Joint Conference on Neural Networks, Baltimore, MD, USA, 1992, pp. 901-906 vol.1

Uwagi

1. Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).

2. This work was supported by VEGA 1/0440/19 research grant.