The application of topological data analysis to human motion recognition

• Autorzy: Żelawski Marcin , Hachaj Tomasz

Identyfikatory

DOI

10.37705/TechTrans/e2021011

• Języki publikacji: EN

Abstrakty

EN

Human motion analysis is a very important research topic in the field of computer vision, as evidenced by a wide range of applications such as video surveillance, medical assistance and virtual reality. Human motion analysis concerns the detection, tracking and recognition of human activities and behaviours. The development of low-cost range sensors enables the precise 3D tracking of body position. The aim of this paper is to present and evaluate a novel method based on topological data analysis (TDA) for motion capture (kinematic) processing and human action recognition. In contrast to existing methods of this type, we characterise human actions in terms of topological features. The recognition process is based on topological persistence which is stable to perturbations. The advantages of TDA are noise resistance and the ability to extract global structure from local information. The method we proposed in this paper deals very effectively with the task of human action recognition, even on the difficult classes of motion found in karate techniques. In order to evaluate our solution, we have performed three-fold cross-validation on a data set containing 360 recordings across twelve motion classes. The classification process does not require the use of machine learning and dynamical systems theory. The proposed classifier achieves a total recognition rate of 0.975 and outperforms the state-of-theart methods (Hachaj, 2019) that use support vector machines and principal component analysis-based feature generation.

Słowa kluczowe

EN

human action classification; topological data analysis; motion capture; martial arts; karate

PL

klasyfikacja ludzkich działań; analiza danych topologicznych; przechwytywanie ruchu; sztuki walki; karate

• Wydawca: Wydawnictwo Politechniki Krakowskiej im. Tadeusza Kościuszki
• Czasopismo: Technical Transactions
• Rocznik: 2021
• Tom: Vol. 118, iss. 1
• Strony: art. no. e2021011
• Opis fizyczny: Bibliogr. 35 poz., tab., wz., wykr.

Twórcy

autor

Żelawski Marcin

• Institute of Computer Science, Pedagogical University of Cracow

autor

Hachaj Tomasz

• Institute of Computer Science, Pedagogical University of Cracow

Bibliografia

• 1. Anirudh, R., Venkataraman, V., Natesan Ramamurthy, K., & Turaga, P. (2016). A riemannian framework for statistical analysis of topological persistence diagrams. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 68–76).
• 2. Bhaskar, D., Manhart, A., Milzman, J., Nardini, J. T., Storey, K. M., Topaz, C. M., & Ziegelmeier, L. (2019). Analyzing collective motion with machine learning and topology. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(12), 123–125.
• 3. Billon, R., Nédélec, A., & Tisseau, J. (2008). Gesture recognition in flow based on PCA and using multiagent system. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology (pp. 239–240).
• 4. Bottino, A., De Simone, M., & Laurentini, A. (2007). Recognizing human motion using eigensequences. Journal of WSCG, 15, 135–142.
• 5. Choi, W., Li, L., Sekiguchi, H., & Hachimura, K. (2013). Recognition of gait motion by using data mining. In 2013 13th International Conference on Control, Automation and Systems (ICCAS 2013) (pp. 1213–1216). IEEE.
• 6. Choi, W., Ono, T., & Hachimura, K. (2009). Body Motion Analysis for Similarity Retrieval of Motion Data and Its Evaluation. In 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (pp. 1177–1180). IEEE.
• 7. Choi, W., Sekiguchi, H., & Hachimura, K. (2009). Analysis of Gait Motion by Using Motion Capture in the Japanese Traditional Performing Arts. In 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (pp. 1164–1167). IEEE.
• 8. Cornacchia, M., Ozcan, K., Zheng, Y., & Velipasalar, S. (2016). A survey on activity detection and classification using wearable sensors. IEEE Sensors Journal, 17(2), 386–403.
• 9. Das, S. R., Wilson, R. C., Lazarewicz, M. T., & Finkel, L. H. (2006). Two-stage PCA extracts spatiotemporal features for gait recognition. Journal of multimedia, 1(5), 9–17.
• 10. Dirafzoon, A., Lokare, N., & Lobaton, E. (2016). Action classification from motion capture data using topological data analysis. In 2016 IEEE global conference on signal and information processing (globalSIP) (pp. 1260–1264). IEEE.
• 11. Edelsbrunner, H., & Harer, J. (2010). Computational topology: an introduction. American Mathematical Soc.
• 12. Edelsbrunner, H., Letscher, D., & Zomorodian, A. (2000). Topological persistence and simplification. Discrete & Computational Geometry, 28(4), 511–533.
• 13. Funakoshi, G. (1996). Karate-Do Kyo-han; The Master Text. Kodansha America LLC.
• 14. Ghrist, R. (2008). Barcodes: the persistent topology of data. Bulletin of the American Mathematical Society, 45(1), 61–75.
• 15. Github (online). https://github.com/browarsoftware/MoCapEigen/tree/master/ data [access: 17/12/2020].
• 16. Hachaj, T. (2019). Improving Human Motion Classification by Applying Bagging and Symmetry to PCA-Based Features. Symmetry, 11(10), 1264.
• 17. Hachaj, T., & Ogiela, M. R. (2018). Classification of Karate Kicks with Hidden Markov Models Classifier and Angle-Based Features. In 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) (pp. 1–5). IEEE.
• 18. Hachaj, T., Piekarczyk, M., & Ogiela, M. R. (2017). Human actions analysis: Templates generation, matching and visualization applied to motion capture of highly-skilled karate athletes. Sensors, 17(11), 2590.
• 19. Idris, W. M. R. W., Rafi, A., Bidin, A., Jamal, A. A., & Fadzli, S. A. (2019). A systematic survey of martial art using motion capture technologies: the importance of extrinsic feedback. Multimedia Tools and Applications, 78(8), 10113–10140.
• 20. Kim, H. C., Kim, D., & Bang, S. Y. (2002). Face recognition using the mixture-of-eigenfaces method. Pattern Recognition Letters, 23(13), 1549–1558.
• 21. Ko, J. H., Han, D. W., & Newell, K. M. (2018). Skill level changes the coordination and variability of standing posture and movement in a pistol-aiming task. Journal of Sports Sciences, 36(7), 809–816.
• 22. Lee, M., Roan, M., & Smith, B. (2009). An application of principal component analysis for lower body kinematics between loaded and unloaded walking. Journal of biomechanics, 42(14), 2226–2230.
• 23. Mantovani, G., Ravaschio, A., Piaggi, P., & Landi, A. (2010). Fine classification of complex motion pattern in fencing. Procedia Engineering, 2(2), 3423–3428.
• 24. Mokari, M., Mohammadzade, H., Ghojogh, B. (2020). Recognizing involuntary actions from 3D skeleton data using body states. Scientia Iranica, 27(3), 1424–1436.
• 25. Mrozek, M., Żelawski, M., Gryglewski, A., Han, S., & Krajniak, A. (2012). Homological methods for extraction and analysis of linear features in multidimensional images. Pattern Recognition, 45(1), 285–298.
• 26. Presti, L. L., & La Cascia, M. (2016). 3D skeleton-based human action classification: A survey. Pattern Recognition, 53, 130–147.
• 27. Som, A., Thopalli, K., Natesan Ramamurthy, K., Venkataraman, V., Shukla, A., & Turaga, P. (2018). Perturbation robust representations of topological persistence diagrams. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 617–635).
• 28. Świtoński, A., Mucha, R., Danowski, D., Mucha, M., Polański, A., Cieślar, G., & Sieroń, A. (2011). Diagnosis of the motion pathologies based on a reduced kinematical data of a gait. Przegląd Elektrotechniczny, 87(12), 173–176.
• 29. Tralie, C. (2016). High-dimensional geometry of sliding window embeddings of periodic videos. In 32nd International Symposium on Computational Geometry (SoCG 2016). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
• 30. Tralie, C. J., & Berger, M. (2018). Topological eulerian synthesis of slow motion periodic videos. In 2018 25th IEEE International Conference on Image Processing (ICIP) (pp. 3573–3577). IEEE.
• 31. Umeda, Y. (2017). Time series classification via topological data analysis. Information and Media Technologies, 12, 228–239.
• 32. Vejdemo-Johansson, M., Pokorny, F. T., Skraba, P., & Kragic, D. (2015). Cohomological learning of periodic motion. Applicable algebra in engineering, communication and computing, 26(1–2), 5–26.
• 33. Venkataraman, V., Ramamurthy, K. N., & Turaga, P. (2016). Persistent homology of attractors for action recognition. In 2016 IEEE international conference on image processing (ICIP) (pp. 4150–4154). IEEE.
• 34. Zago, M., Pacifici, I., Lovecchio, N., Galli, M., Federolf, P. A., & Sforza, C. (2017). Multi-segmental movement patterns reflect juggling complexity and skill level. Human Movement Science, 54, 144–153.
• 35. Zomorodian, A., & Carlsson, G. (2005). Computing persistent homology. Discrete & Computational Geometry, 33(2), 249–274.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).