British sign language recognition in the wild based on multi-class SVM

• Autorzy: Olszewska Joanna I. , Quinn M.

Identyfikatory

DOI

10.15439/2019F274

• Konferencja: Federated Conference on Computer Science and Information Systems (14 ; 01-04.09.2019 ; Leipzig, Germany)
• Języki publikacji: EN

Abstrakty

EN

Developing assistive, cost-effective, non-invasive technologies to aid communication of people with hearing impairments is of prime importance in our society, in order to widen accessibility and inclusiveness. For this purpose, we have developed an intelligent vision system embedded on a smartphone and deployed in the wild. In particular, it integrates both computer vision methods involving Histogram of Oriented Gradients (HOG) and machine learning techniques such as multiclass Support Vector Machine (SVM) to detect and recognize British Visual Language (BSL) signs automatically. Our system was successfully tested on a real-world dataset containing 13,066 samples and shown an accuracy of over 99% with an average processing time of 170ms, thus appropriate for real-time visual signing.

Słowa kluczowe

EN

computer vision; handicapped aids; hearing; artificial intelligence; sign language recognition; smartphone; support vector machine

PL

wizja komputerowa; osoby z niepełnosprawnością; sztuczna inteligencja; rozpoznawanie języka migowego; smartfon; maszyna wektorów nośnych

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2019
• Tom: Vol. 18
• Strony: 81--86
• Opis fizyczny: Bibliogr. 21 poz., il., wykr., wz.

Twórcy

autor

Olszewska Joanna I.

• School of Computing and Engineering, University of the West of Scotland, United Kingdom

autor

Quinn M.

• School of Computing and Engineering, University of the West of Scotland, United Kingdom

Bibliografia

• 1. Actiononhearingloss.org.uk, “Facts and Figures,,” 2019, Available online at: https://www.actiononhearingloss.org.uk.
• 2. BDA, “British Deaf Association,,” 2019, Available online at: https://bda.org.uk.
• 3. NRCPD, “The National Registers of Communication Professionals working with Deaf and Deafblind People,,” 2019, Available online at: https://www.nrcpd.org.uk.
• 4. D. Waters, R. Campbell, C.M. Capek, B. Woll, A.S. David, P.K. McGuire, M.J. Brammer, and M. MacSweeney, “Fingerspelling, signed language, text and picture processing in deaf-native signers: The role of the mid-fusiform gyrus,” NeuroImage, vol. 35, no. 3, pp. 1287–1302, 2007.
• 5. J.I. Olszewska, “Designing transparent and autonomous intelligent vision systems,” in Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART), 2019, pp. 850–856.
• 6. G. Plouffe and A.-M. Cretu, “Static and dynamic hand gesture recognition in depth data using dynamic time warping,” IEEE Transactions on Instrumentation and Measurement, vol. 65, no. 2, pp. 305–316, 2016.
• 7. M. Goyal, B. Shahi, K.V. Prema, and N.V.S.S. Reddy, “Performance analysis of human gesture recognition techniques,” in Proceedings of the IEEE International Conference on Recent Trends in Electronics, Information and Communication Technology, 2017, pp. 111–115.
• 8. S. Liwicki and M. Everingham, “Automatic recognition of fingerspelled words in British sign language,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 50–57.
• 9. J.L. Raheja, A. Mishra, and A. Chaudhary, “Indian sign language recognition using SVM,” Pattern Recognition and Image Analysis, vol. 26, no. 2, pp. 434–441, 2016.
• 10. A.B. Jani, N.A. Kotak, and A.K. Roy, “Sensor based hand gesture recognition system for English alphabets used in sign language of deaf-mute people,” in IEEE SENSORS Proceedings, 2018, pp. 1–4.
• 11. D.-J. Li an Y.-Y. Li, J.-X. Li, and Y. Fu, “Gesture recognition based on BP neural network improved by chaotic genetic algorithm,” International Journal of Automation and Computing, vol. 15, no. 3, pp. 267–276, 2018.
• 12. S. Salian, I. Dokare, D. Serai, A. Suresh, and P. Ganorkar, “Proposed system for sign language recognition,” in Proceedings of the IEEE Conference on Computation of Power, Energy Information and Communication, 2017, pp. 58–62.
• 13. B.L. Loeding, S. Sarkar, A. Parashar, and A.I. Karshmer, “Progress in automated computer recognition of sign language,” in Proceedings of the International Conference on Computers for Handicapped Persons. 2004, pp. 1079–1087, LNCS, Springer.
• 14. E. Klima and U. Bellugi, The Signs of Language, Harvard University Press, 1979.
• 15. M. Loesdau, S. Chabrier, and A. Gabillon, “Hue and saturation in the RGB color space,” in Proceedings of the International Conference on Image and Signal Processing. 2014, vol. 8509, pp. 203–212, LNCS, Springer.
• 16. C. Rouge, S. Shaikh, and J.I. Olszewska, “HD: Efficient Hand Detection and Tracking,” in Proceedings of the IEEE Federated Conference on Computer Science and Information Systems (FedCSIS), 2016, pp. 291–297.
• 17. J.I. Olszewska, C. De Vleeschouwer, and B. Macq, “Multi-feature vector flow for active contour tracking,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2008, pp. 721–724.
• 18. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005, pp. 886–893.
• 19. C. Cortes and V.N. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.
• 20. P. Matykiewicz and J. Pestian, “Effect of small sample size on text categorization with support vector machines,” in Proceedings of the ACL Workshop on Biomedical Natural Language Processing (BioNLP), 2012, pp. 193–201.
• 21. S. Nagarajan and T.S. Subashini, “Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi-class SVM,” International Journal of Computer Applications, vol. 82, no. 4, pp. 28–35, 2013.

Uwagi

1. Track 1: Artificial Intelligence and Applications

2. Technical Session: 14th International Symposium Advances in Artificial Intelligence and Applications

3. Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).