Human gesture recognition using hidden Markov models and sensor Fusion

• Autorzy: Emmanuel Dominguez Ramon , Raquel Diaz Hernandez , Leopoldo Altamirano Robles

Identyfikatory

DOI

10.7494/csci.2022.23.2.3745

• Języki publikacji: EN

Abstrakty

EN

Considering the continued drive of human needs along with the constant improvement of technology, it is convenient to develop techniques that can enhance communication between computers and humans in the most intuitive ways possible. The possibility of automatically recognizing human gestures using artificial vision (among other kinds of sensors) allows us to explore a whole range of applications to control and interact with environments. Nowadays, most approaches for gesture recognition using sensors agree in the use of vision, myography, and movement devices that are applied to robotic, medical, and industrial applications. In the context of this work, we study the principles of using both vision and body contact sensing applied to the automatic classification of a human gesture set. For this, two different approaches have been evaluated: feed-forward neural networks, and hidden Markov models. These models have been studied and implemented for recognizing up to eight different human hand gestures that are commonly applied in collaborative robotics tasks.

Słowa kluczowe

EN

gesture recognition; collaborative robotics; vision-based sensors; myography sensors; sensor fusion

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2022
• Tom: T. 23 (2)
• Strony: 227--245
• Opis fizyczny: Bibliogr. 27 poz., rys., tab.

Twórcy

autor

Emmanuel Dominguez Ramon

• INAOE, Mexico

autor

Raquel Diaz Hernandez

• INAOE, Mexico

autor

Leopoldo Altamirano Robles

• INAOE, Mexico

Bibliografia

• [1] Ahmad M., Lee S.W.: HMM-based human action recognition using multiview image sequences. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 263–266, IEEE, 2006.
• [2] Ahsan M.R., Ibrahimy M.I., Khalifa O.O.: Electromygraphy (EMG) signal based hand gesture recognition using artificial neural network (ANN). In: 2011 4th International Conference on Mechatronics (ICOM), pp. 1–6, IEEE, 2011.
• [3] Athavale S., Deshmukh M.: Dynamic Hand Gesture Recognition for Human Computer interaction; A Comparative Study, International Journal of Engineering Research and General Science, vol. 2(2), pp. 38–55, 2014.
• [4] Benalcazar M.E., Anchundia C.E., Zea J.A., Zambrano P., Jaramillo A.G., Segura M.: Real-Time Hand Gesture Recognition Based on Artificial Feed-Forward Neural Networks and EMG. In: 2018 26th European Signal Processing Conference (EUSIPCO), pp. 1492–1496, IEEE, 2018.
• [5] Figueroa Y.S.: Reconocimiento anticipado de gestos, Instituto Nacional de Astrof´ısica, Optica y Electrˆenica, 2013. http://inaoe.repositorioinstitucional.mx/jspui/handle/1009/803.
• [6] Georgi M., Amma C., Schultz T.: Recognizing Hand and Finger Gestures with IMU based Motion and EMG based Muscle Activity Sensing. In: Biosignals, pp. 99–108, 2015.
• [7] Jin H., Chen Q., Chen Z., Hu Y., Zhang J.: Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task, CAAI Transactions on Intelligence Technology, vol. 1(1), pp. 104–113, 2016.
• [8] Jorgensen S.J.M.: Human detection, gesture recognition, and policy generation for human-aware robots, Ph.D. thesis, The University of Texas at Austin, 2017.
• [9] Junker H., Amft O., Lukowicz P., Troster G.: Gesture spotting with bodyworn inertial sensors to detect user activities, Pattern Recognition, vol. 41(6), pp. 2010–2024, 2008.
• [10] Kılıboz N.C¸., Gudukbay U.: A hand gesture recognition technique for human–computer interaction, Journal of Visual Communication and Image Representation, vol. 28, pp. 97–104, 2015. 244 Domınguez Ramon Emmanuel, Dıaz Hernandez Raquel, Altamirano Robles Leopoldo
• [11] Lake S., Bailey M., Grant A.: Method and apparatus for analyzing capacitive EMG and IMU sensor signals for gesture control, US Patent: 09299248, 2016. https://patentscope.wipo.int/search/en/detail.jsf?docId=US107210802.
• [12] Lee K.C., Kweon J.H., Kim K.J., Choi J.W.: Method and apparatus for controlling a home device remotely in a home network system, US Patent: 09978260, 2018. https://patentscope.wipo.int/search/en/detail.jsf?docId=US105465928.
• [13] Lv F., Nevatia R.: Recognition and segmentation of 3-d human action using HMM and multi-class AdaBoost. In: European Conference on Computer Vision, pp. 359–372, Springer, 2006.
• [14] Mitra S., Acharya T.: Gesture recognition: A survey, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37(3), pp. 311–324, 2007.
• [15] Molchanov P., Gupta S., Kim K., Pulli K.: Multi-sensor system for driver’s handgesture recognition. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–8, IEEE, 2015.
• [16] Nagar P., Sengar A., Sharma M.: Hand shape based gesture recognition in hardware, Archives of Applied Science Research, vol. 5(3), pp. 261–269, 2013.
• [17] Obaid M., Kistler F., Kasparaviciute G., Yantac A.E., Fjeld M.: How would you gesture navigate a drone?: a user-centered approach to control a drone. In: Proceedings of the 20th International Academic Mindtrek Conference, pp. 113–121, ACM, 2016.
• [18] Pink O., Becker J., Kammel S.: Automated driving on public roads: Experiences in real traffic, it-Information Technology, vol. 57(4), pp. 223–230, 2015.
• [19] Premaratne P., Yang S., Vial P., Ifthikar Z.: Centroid tracking based dynamic hand gesture recognition using discrete hidden Markov models, Neurocomputing, vol. 228, pp. 79–83, 2017.
• [20] Rabiner L., Juang B.: An introduction to hidden Markov models, IEEE ASSP Magazine, vol. 3(1), pp. 4–16, 1986.
• [21] Rautaray S.S., Agrawal A.: Vision based hand gesture recognition for human computer interaction: a survey, Artificial Intelligence Review, vol. 43(1), pp. 1–54, 2015.
• [22] Rautiainen T.T., Hui P., Kaunisto R.H.S., Teikari I.A., Ollikainen J.P.J.: Gesture control, US Patent: 09335825, 2016. https://patentscope.wipo.int/search/en/detail.jsf?docId=US73307542.
• [23] Sinha K., Kumari R., Priya A., Paul P.: A Computer Vision-Based Gesture Recognition Using Hidden Markov Model. In: Innovations in Soft Computing and Information Technology, pp. 55–67, Springer, 2019.
• [24] Sun Y., Li C., Li G., Jiang G., Jiang D., Liu H., Zheng Z., Shu W.: Gesture recognition based on kinect and sEMG signal fusion, Mobile Networks and Applications, vol. 23(4), pp. 797–805, 2018. Human gesture recognition using hidden Markov models and sensor fusion 245
• [25] Wang W., Li R., Diekel Z.M., Chen Y., Zhang Z., Jia Y.: Controlling Object Hand-Over in Human–Robot Collaboration Via Natural Wearable Sensing, IEEE Transactions on Human-Machine Systems, vol. 49(1), pp. 59–71, 2018.
• [26] Yu T., Finn C., Xie A., Dasari S., Zhang T., Abbeel P., Levine S.: One-shot imitation from observing humans via domain-adaptive meta-learning, arXiv preprintarXiv:180201557, 2018.
• [27] Zabulis X., Baltzakis H., Argyros A.A.: Vision-Based Hand Gesture Recognition for Human-Computer Interaction., The Universal Access Handbook, vol. 34, p. 30, 2009.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023).