Ocena modeli rozpoznawania pozy ręki wykorzystujących głębokie sieci neuronowe

• Autorzy: Zubrzycki Igor , Wieteska Anna

Warianty tytułu

EN

Assessment of hand pose recognition models based on deep neural networks in a unified environment

• Języki publikacji: PL

Abstrakty

PL

W ostatnich latach powstały nowoczesne modele uczenia maszynowego oparte o sieci neuronowe, pozwalające na określenie punktów charakterystycznych oraz pozy ręki. Konkursy takie jak HANDS 2019 pozwoliły określić najlepsze z nich (SOTA), jednak brakuje jednolitego środowiska pozwalającego na porównanie ich do siebie, np. w celu określenia zachowania tych modeli w szczególnych warunkach i dostosowanie sieci do własnych potrzeb. W pracy przedstawiamy krótki przegląd istniejących podejść i zbiorów danych oraz opisujemy działania mające na celu stworzenie standaryzowanego środowiska do uruchamiania modeli regresji punktów charakterystycznych ręki. Opisujemy potencjalne zastosowanie – porównanie modeli do zastosowania ich przy modelowaniu ręki sparaliżowanej w celu opracowania personalizowanego robota miękkiego.

EN

The paper presents a short review of modern models for hand key-point detection based on depth data. We designed a unified containerized system for their comparison and ease of use. We tested the setup and re-evaluated the models’ behaviour on the NYU hand pose dataset. The future goal of the work is to use a model with the best results for hand modelling for a personalized design of rehabilitation devices.

Słowa kluczowe

PL

systemy wizyjne; systemy sensoryczne; ręka; sieci neuronowe

EN

vision systems; sensory system; hand; neural networks

• Wydawca: Oficyna Wydawnicza Politechniki Warszawskiej
• Czasopismo: Prace Naukowe Politechniki Warszawskiej. Elektronika
• Rocznik: 2022
• Tom: z. 197, t. 2
• Strony: 77--88
• Opis fizyczny: Bibliogr. 29 poz., rys., wykr. tab.

Twórcy

autor

Zubrzycki Igor

• Zakład Sterowania Robotów, Instytut Automatyki, Politechnika Łódzka

autor

Wieteska Anna

• Zakład Sterowania Robotów, Instytut Automatyki, Politechnika Łódzka

Bibliografia

• 1. Anil Armagan et al. Measuring generalisation to unseen viewpoints, articulations, shapes and objects for 3d hand pose estimation under hand-object interaction. In: Proceedings of the European Conference on Computer Vision (ECCV). Proceedings, 2020.
• 2. Anil Armagan et al. Measuring generalisation to unseen viewpoints, articulations, shapes and objects for 3d hand pose estimation under hand-object interaction. In: Computer Vision – ECCV 2020, s. 85-101. Springer International Publishing 2020.
• 3. Valentin Bazarevsky, Fan Zhang. On-device, real-time hand tracking with media-pipe. Google AI Blog, 2019.
• 4. Zhe Cao et al. Realtime multi-person 2d pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, July, 2017.
• 5. Xiaoming Deng et al. Hand3d: Hand pose estimation using 3d neural network, 2017.
• 6. Farnaz Farahanipad et al. A pipeline for hand 2-d keypoint localization using unpaired image to image translation. In: The 14th Pervasive Technologies Related to Assistive Environments Conference. Proceedings. ACM, jun, 2021.
• 7. Guillermo Gracia-Hernando et al. First-person hand action benchmark with rgb-d videos and 3d hand pose annotations. In: Proceedings of the IEEE Conference on computer Vision and Pattern Recognition (CVPR). Proceedings, June, 2018.
• 8. Liuhao et al. 3d convolutional neural networks for efficient and robust hand pose estimation from single depth images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, July, 2017.
• 9. Shreyas Hampali et al. Honnoate: A method for 3d annotation of hand and object poses. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, June, 2020.
• 10. Weiting Huang et al. AWR: adaptive weighting regression for 3d hand pose estimation. CoRR, 2020, wolumen abs/2007.09590.
• 11. Rui Li, Zhenyu Liu, Jianrong Tan. A survey on 3d hand pose estimation: Cameras, methods, and datasets. Pattern Recognition, sep, 2019, wolumen 93, s. 251-272.
• 12. Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee. V2V-PoseNet, matlab preprocessing file. https://github.com/mks0601/V2V-PosetNet_RELEASE/blob/master/data/NYU/PNG2BIN.m. Accessed: 2021-01-27.
• 13. Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee. V2V-posenet: Voxel-to-voxel prediction network for accurate 3d hand and human pose estimation from a single dpth map. CoRR, 2017, wolumen abs/1711.07399.
• 14. Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee. V2V-posenet: Voxel-to-voxel prediction network for accurate 3d hand and human pose estimation from a single dpth map. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, June, 2018.
• 15. Alejandro Newell, Kaiyu Yang, Jia Deng. Stacked hourglass networks for human pose estimation. In: Computer Vision – ECCV 2016, s. 483-499. Springer International Publishing 2016.
• 16. Nvidia. Nvidia Tensorflow documentation. https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/running.html. Accessed: 2021-01-28.
• 17. Adam Paszke et al. Pythorch documentation. https://pytorch.org/. Accessed: 2021-02-09.
• 18. Chen Qian et al. Realtime and robust hand tracking from depth. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Proceedings, 2014, s. 1106-1113.
• 19. Javier Romero, Dimitrios Tzionas, Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together. CoRR, 2022, wolumen abs/2201.02610.
• 20. Jonathan Tompson et al. Real-time continuous pose recovery of human hands using convolutional networks. AC Transactions on Graphics, August, 2014, wolumen 33.
• 21. Shij-En Wei et al. Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, June, 2016.
• 22. Bin Xiao, Haiping Wu, Yichen Wei. Simple baselines for human pose estimation and tracking. In: Proceedings of the European Conference on Computer Vision (ECCV). Proceedings, September, 2018.
• 23. Fu Xiong et al. A2J, matlab preprocessing file. https://github.com/zhangboshen/A2J/blob/master/data/nyu/data_preprosess.m Accessed: 2021-01-27.
• 24. Fu Xiong et al. A2J: Anchor-to-joint regression network for 3d articulated pose estimation from a single depth image. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Proceedings, October, 2019.
• 25. Fu Xiong et al. A2J: Anchor-to-joint regression network for 3d articulated pose estimation from a single depth image. CoRR, 2019, wolumen abs/1908.09999.
• 26. Shanxin Yuan et al. Bighand2.2m benchmark: Hand pose dataset and state of the art analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings, July, 2017.
• 27. Zhaohui Zhang et al. HandAugment: A Simple Data Augmentation Method for Depth-Based 3D Hand Pose Estimation, papers with code. https://paperswithcode.com/paper/handaugment-a-simple-data-augmentation-for. Accessed: 2021-02-09.
• 28. Zhaohui Zhang et al. Handaugment: A simple data augmentation method for depth-based 3d hand pose estimation. CoRR, 2020, wolumen abs/2001.00702.
• 29. Christian Zimmermann, Thomas Brox. Learning to estimate 3d hand pose from single rgb images. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Proceedings, Oct, 2017.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).