Tennis Patterns Recognition Based on a Novel Tennis Dataset – 3DTennisDS

• Autorzy: Skublewska-Paszkowska Maria , Powroznik Paweł , Lukasik Edyta , Smolka Jakub

Identyfikatory

DOI

10.12913/22998624/191264

• Języki publikacji: EN

Abstrakty

EN

Many scientific studies on tennis stroke recognition are based on datasets created for the purpose of research using video or motion capture techniques. The importance of such datasets has been increasing due to the athlete performance evaluation needs. The primary aim of this paper is to present a state-of-the-art 3DTennisDS storing four tennis strokes: forehand, backhand, volley forehand and volley backhand. The moves were registered using the Vicon optical motion capture and contain a 39-marker player and a 7-marker tennis racket models. The potential and quality of this unique dataset has been verified using Spatial-Temporal Graph Neural Networks, because this type of network topology matches to the human body structure. The presented 3DTennisDS has been compared with two well-known datasets: the THETIS and the Tennis-Mocap. They contain tennis movements in a form of motion capture data, registered using markerless and marker-based systems. The classification of tennis strokes has been performed to verify how various types of data acquisition (marker-based and marker-less ones) as well as the structure of the data affect the accuracy of human action recognition. In this study ONI files from THETIS, bvh from Tennis-Mocap and c3d data from 3DTennisDS were considered. Moreover, the impact of input data fuzzification was examined. The obtained results showed that the classification using 3DTennisDS achieved the best results, both for fuzzy and non-fuzzy inputs. These outcomes indicate that the way of capturing data, its preparation and structure have great influence on classification accuracy. The developed 3DTennisDS has a great potential in further motion capture analysis.

Słowa kluczowe

EN

3DTennisDS; tennis dataset; tennis strokes; motion capture; graph convolutional network; fuzzy classification

• Wydawca: Lublin University of Technology ; Polish Society of Ecological Engineering (PTIE), Branch of PTIE in Lublin
• Czasopismo: Advances in Science and Technology. Research Journal
• Rocznik: 2024
• Tom: Vol. 18, no 6
• Strony: 159--176
• Opis fizyczny: Bibliogr. 53 poz., fig., tab.

Twórcy

autor

Skublewska-Paszkowska Maria

• Department of Computer Science, Lublin University of Technology, ul. Nadbystrzycka 38D, 20-618 Lublin, Poland

• https://orcid.org/0000-0002-0760-7126

autor

Powroznik Paweł

• Department of Computer Science, Lublin University of Technology, ul. Nadbystrzycka 38D, 20-618 Lublin, Poland

• https://orcid.org/0000-0002-5705-4785

autor

Lukasik Edyta

• Department of Computer Science, Lublin University of Technology, ul. Nadbystrzycka 38D, 20-618 Lublin, Poland

• https://orcid.org/0000-0003-3644-9769

autor

Smolka Jakub

• Department of Computer Science, Lublin University of Technology, ul. Nadbystrzycka 38D, 20-618 Lublin, Poland

• https://orcid.org/0000-0002-0760-7126

Bibliografia

• 1. Rajšp A., Fister Jr, I. A systematic literature review of intelligent data analysis methods for smart sport training. Appl. Sci. 2020, 10(9): 3013.
• 2. Tennis Creative. Tennis Popularity Statistics 2021. https://tenniscreative.com/tennis-popularity-statistics/, 2022, [Online: 9.05.2024].
• 3. Vincent J., Stergiou P., Katz L. The role of databases in sport science: Current practice and future potential. International Journal of Computer Science in Sport. 2009, 8(2): 50–66.
• 4. Zhang W., Liu Z., Zhou L., Leung H., Chan A. Martial arts, dancing and sports dataset: A challenging stereo and multi-view dataset for 3d human pose estimation. Image Vis. Comput. 2017, 61: 22–39.
• 5. Gourgari S., Goudelis G., Karpouzis K., Kollias S. Thetis: Three dimensional tennis shots a human action dataset. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013, 676–681.
• 6. Cai J., Hu J., Tang X., Hung T.-Y., Tan Y.-P. Deep historical long short-term memorys for action recognition. Neurocomputing. 2020, 407: 428–438.
• 7. Sen A., Hossain S. M., Uddin R., Deb K, Jo K.-H. Sequence recognition of indoor tennis actions using transfer learning and long short-term memory. In: International Workshop on Frontiers of Computer Vision, Springer, 2022, 312–324.
• 8. Mora S.V., Knottenbelt W.J. Deep learning for domain-specific action recognition in tennis. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, 170–178.
• 9. Mora S. Vision and machine learning for in-play tennis analysis: framework, algorithms and implementation. Imperial College of London, Technology and Medicine, 2017.
• 10. Ullah M., Mudassar Yamin M., Mohammed A., Daud Khan S., Ullah H., Alaya Cheikh F. Attentionbased lstm network for action recognition in sports. J. Electron. Imaging. 2021, 33: 1–6.
• 11. Vainstein J. Manera J., Negri P., Delrieux C., Maguitman A. Modeling video activity with dynamic phrases and its application to action recognition in tennis videos. In: n Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 19th Iberoamerican Congress, CIARP 2014, Puerto Vallarta, Mexico, November 2–5, 2014. Proceedings 19, 909–916. Springer International Publishing.
• 12. Tsatiris G., Karpouzis K., Kollias S. Variance-based shape descriptors for determining the level of expertise of tennis players. In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games). 2017, 169–172.
• 13. Chang H. Research on tennis motion evaluation method and system based on deep learning. Int T Electr Energy. 2022.
• 14. Valencia-Marin C. K., Pulgarin-Giraldo J. D., Velasquez-Martinez L. F., Alvarez-Meza A. M,. Castellanos-Dominguez G. An enhanced joint Hilbert embedding-based metric to support mocap data classification with preserved interpretability. Sensors. 2021, 21(13): 4443.
• 15. Yamada K., Matsuura M., Hamagami K., Inui H.Motor skill development using motion recognition based on an hmm. Procedia Comput. Sci. 2013, 22: 1112–1120.
• 16. Bačić B. Towards the next generation of exergames: Flexible and personalised assessment-based identification of tennis swings. In: 2018 International Joint Conference on Neural Networks. IEEE. 2018, 1–8.
• 17. Bezobrazov S., Sheleh A., Kislyuk S., Golovko V., Sachenko A., Komar M., Dorosh V., Turchenko V. Artificial intelligence for sport activitity recognition. In: 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), IEEE. 2019, 2: 628–632.
• 18. Ganser A., Hollaus B., Stabinger S. Classification of tennis shots with a neural network approach. Sensors. 2021, 21: 17.
• 19. Ma K. A real time artificial intelligent system for tennis swing classification. In: 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), IEEE. 2021, 000 021–000 026.
• 20. Ó Conaire C., Connaghan D., Kelly P., O’Connor N. E., Gaffney M., Buckley J. Combining inertial and visual sensing for human action recognition in tennis. In: Proceedings of the first ACM international workshop on Analysis and retrieval of tracked events and motion in imagery streams. 2010, 51–56.
• 21. Skublewska-Paszkowska M., Powroznik P., Lukasik E. Learning three dimensional tennis shots using graph convolutional networks. Sensors. 2020, 20(2): 6094.
• 22. Weinzaepfel P., Rogez G. Mimetics: Towards understanding human actions out of context. Int. J. Comput. Vis. 2021, 129: 1675–1690.
• 23. Kviatkovsky I., Rivlin E., Shimshoni I. Online action recognition using covariance of shape and motion. Comput. Vis. Image Underst. 2014, 129: 15–26.
• 24. Soomro K., Zamir A., Shah M. Ucf101: A dataset of 101 human actions classes from videos in the wild. CoRR. 2012, 12.
• 25. López-Méndez A., Alcoverro M., Pardàs M., Casas J.R. Approximate partitioning of observations in hierarchical particle filter body tracking. In: Cvpr 2011 Workshops. IEEE. 2011, 19–24.
• 26. Matsugu M., Yamanaka M., Sugiyama M. Detection of activities and events without explicit categorization. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 2011, 1532–1539.
• 27. Tsagkatakis G., Jaber M., Tsakalides P. Convolutional neural networks for the analysis of broadcasted tennis games. Electron. Imagin. 2018, 206–2011.
• 28. Kurose R., Hayashi M., Aoki Y. Estimation of scenes contributing to score in tennis video using attention. In: 2018 14th International Conference on Signal-Image Technology and Internet-Based Systems (SITIS). 2018, 672–675.
• 29. Sukhwani M. Jawahar C. Frame level annotations for tennis videos. In: 2016 23rd International Conference on Pattern Recognition (ICPR). 2016, 841–846.
• 30. Sukhwani M. Jawahar C.V. Tennisvid2text: Finegrained descriptions for domain specific videos. ArXiv. 2015, abs/1511.08522.
• 31. Shimizu T., Hachiuma R., Saito H., Yoshikawa T., Lee C. Prediction of future shot direction using pose and position of tennis player. 2019, 10: 59–66.
• 32. Connaghan D., O’Conaire C., Kelly P., O’Connor N. E. Recognition of tennis strokes using key postures. In: IET Irish Signals and Systems Conference (ISSC 2010). 2010, 245–248.
• 33. Kobayashi S., Kaseda H., Miyamoto R. Robust localization of body parts based on interframe failure correction. In: 2018 IEEE International Conference on Systems, Man, and Cybernetics. 2018, 3903–3908.
• 34. Faulkner H., Dick A. Tenniset: A dataset for dense fine-grained event recognition, localisation and description. In: 2017 International Conference on Digital Image Computing: Techniques and Applications. 2017, 1–8.
• 35. Huang Y.-C., Liao I.-N., Chen C.-H., Ik T.-U., Peng W.-C. Tracknet: A deep learning network for tracking high-speed and tiny objects in sports applications. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance. IEEE. 2019, 1–8.
• 36. Maheswari U., Ramakrishnan R. Sports video classification based on the distribution of edge strengths of non subsampled shearlet transform. Int. J. Appl. Eng. Res. 2014, 9(23): 21835–21850.
• 37. Ely J., Gauriot R., Page L. Do agents maximise? risk taking on first and second serves in tennis. J. Econ. Psychol. 2017, 63: 135–142.
• 38. Wei X., Lucey P., Morgan S., Carr P., Reid M., Sridharan S. Predicting serves in tennis using style priors. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2015, 2207–2215.
• 39. Qazi T., Mukherjee P., Srivastava S., Lall B., Chauhan N. R. Automated ball tracking in tennis videos. In: 2015 Third International Conference on Image Information Processing. 2015, 236–240.
• 40. Wu J., Liu D., Guo Z., Xu Q., Wu Y. Tacticflow: Visual analytics of ever-changing tactics in racket sports. IEEE Trans. Vis. Comput. Graph. 2022, 28(1): 835–845.
• 41. FarajiDavar N., De Campos T., Kittler J., Yan F. Transductive transfer learning for action recognition in tennis games. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE. 2011, 1548–1553,
• 42. Zhu G., Xu C., Huang Q., Gao W., Xing L. Player action recognition in broadcast tennis video with applications to semantic analysis of sports game. In: Proceedings of the 14th ACM international conference on Multimedia. 2006, 431–440.
• 43. Zhu G., Xu C., Gao W., Huang Q. Action recognition in broadcast tennis video using optical flow and support vector machine. In: Computer Vision in Human-Computer Interaction: ECCV 2006 Workshop on HCI, Graz, Austria, Proceedings 9. Springer, 2006, 89–98.
• 44. Zhu G., Xu C., Huang Q., Gao W. Action recognition in broadcast tennis video. In: 18th International Conference on Pattern Recognition. 2006, 1: 251–254. IEEE.
• 45. Davis R. B., Ounpuu S., Tyburski D., Gage J. R. A gait analysis data collection and reduction technique. Hum. Mov. Sci. 1991, 10(5): 575–587.
• 46. Yan S., Xiong Y., Lin D. Spatial temporal graph convolutional networks for skeleton-based action recognition. In: Thirty-second AAAI conference on artificial intelligence. 2018.
• 47. Dai J., Qi H., Xiong Y., Li Y., Zhang G., Hu H., Wei Y. Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision. 2017, 764–773.
• 48. Kipf T. N., Welling M. Semi-supervised classification with graph convolutional networks. arXiv. 2016, preprint arXiv, 1609.02907.
• 49. Duddu V., Samanta D., Rao D.V. Fuzzy graph modelling of anonymous networks. In: International Workshop Soft Computing Applications. Springer. 2018, 432–444.
• 50. Krleža D., Fertalj K. Graph matching using hierarchical fuzzy graph neural networks. IEEE Trans. Fuzzy Syst. 2016, 25(4): 892–904.
• 51. Minaee S., Boykov Y.Y., Porikli F., Plaza A.J., Kehtarnavaz N., Terzopoulos D. Image segmentation using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2021.
• 52. Pei W., Wang J., Xu X., Wu Z., Du X. An embedded 6-axis sensor based recognition for tennis stroke. In: 2017 IEEE International Conference on Consumer Electronics. IEEE. 2017, 55–58.
• 53. Skublewska-Paszkowska M., Powroznik P., Lukasik E. Attention temporal graph convolutional network for tennis groundstrokes phases classification. In: 2022 IEEE International Conference on Fuzzy Systems. IEEE. 2022, 1–8.