Hand movement disorders tracking by smartphone based on computer vision methods

• Autorzy: Andrushchenko Marko , Selivanova Karina , Avrunin Oleg , Palii Dmytro , Tymchyk Sergii , Turlykozhayeva Dana

Identyfikatory

DOI

10.35784/iapgos.6126

Warianty tytułu

PL

Śledzenie zaburzeń ruchu dłoni za pomocą smartfona w oparciu o metody wizji komputerowej

• Języki publikacji: EN

Abstrakty

EN

This article describes the development of a cost-effective, efficient, and accessible solution for diagnosing hand movement disorders using smartphone-based computer vision technologies. It highlights the idea of using ToF camera data combined with RG data and machine learning algorithms to accurately recognize limbs and movements, which overcomes the limitations of traditional motion recognition methods, improving rehabilitation and reducing the high cost of professional medical equipment. Using the ubiquity of smartphones and advanced computational methods, the study offers a new approach to improving the quality and accessibility of diagnosis of movement disorders, offering a promising direction for future research and application in clinical practice.

PL

W niniejszym artykule opisano opracowanie opłacalnego, wydajnego i dostępnego rozwiązania do diagnozowania zaburzeń ruchu ręki przy użyciu technologii wizyjnych opartych na smartfonach. Podkreślono w nim ideę wykorzystania danych z kamery ToF w połączeniu z danymi RG i algorytmami uczenia maszynowego do dokładnego rozpoznawania kończyn i ruchów, co przezwycięża ograniczenia tradycyjnych metod rozpoznawania ruchu, poprawiając rehabilitację i zmniejszając wysokie koszty profesjonalnego sprzętu medycznego. Wykorzystując wszechobecność smartfonów i zaawansowane metody obliczeniowe, badanie oferuje nowe podejście do poprawy jakości i dostępności diagnostyki zaburzeń ruchu, oferując obiecujący kierunek przyszłych badań i zastosowań w praktyce klinicznej.

Słowa kluczowe

EN

healthcare; information medical technologies; image analysis; computer vision; artificial intelligence; motion disorders

PL

opieka zdrowotna; informatyczne technologie medyczne; analiza obrazu; wizja komputerowa; sztuczna inteligencja; zaburzenia ruchu

• Wydawca: Wydawnictwo Politechniki Lubelskiej
• Czasopismo: Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska
• Rocznik: 2024
• Tom: T. 14, nr 2
• Strony: 5--10
• Opis fizyczny: Bibliogr. 27 poz., fot., tab., wykr.

Twórcy

autor

Andrushchenko Marko

• Kharkiv National University of Radio Electronics, Department of Biomedical Engineering, Kharkiv, Ukraine

• https://orcid.org/0000-0003-1722-2390

autor

Selivanova Karina

• Kharkiv National University of Radio Electronics, Department of Biomedical Engineering, Kharkiv, Ukraine

• https://orcid.org/0000-0003-1002-0761

autor

Avrunin Oleg

• Kharkiv National University of Radio Electronics, Department of Biomedical Engineering, Kharkiv, Ukraine

• https://orcid.org/0000-0002-6312-687X

autor

Palii Dmytro

• National Pirogov Memorial Medical University, Vinnytsya, Ukraine

• https://orcid.org/0000-0001-6537-6912

autor

Tymchyk Sergii

• Vinnytsia National Technical University, Vinnytsia, Ukraine

• https://orcid.org/0000-0003-2977-1602

autor

Turlykozhayeva Dana

• Al-Farabi Kazakh National University, Scientific Research Institute of Experimental and Theoretical Physics, Almaty, Kazakhstan

• https://orcid.org/0000-0002-7326-9196

Bibliografia

• [1] Apple Machine Learning Research (n.d.). Deploying Transformers on the Apple Neural Engine [https://machinelearning.apple.com/research/neuralengine-transformers].
• [2] Apple Inc. (n.d.). Streaming Depth Data from the TrueDepth Camera. Apple Developer Documentation [https://developer.apple.com/documentation/avfoundation/additional_data_capture/streaming_depth_data_from_the_truedepth_camera].
• [3] Apple Developer Documentation. Streaming Depth Data from the TrueDepth Camera | Apple Developer Documentation [https://developer.apple.com/documentation/avfoundation/additional_data_capture/streaming_depth_data_from_the_truedepth_camera,2023] (accessed 3 Dec. 2023).
• [4] Assimp.org. The Asset-Importer Library Home [https://www.assimp.org] (accessed 3 Dec. 2023).
• [5] Avrunin O. G. et al.: Research Active Posterior Rhinomanometry Tomography Method for Nasal Breathing Determining Violations. Sensors 21, 2021, 8508.
• [6] Avrunin O. et al.: Improving the methods for visualization of middle ear pathologies based on telemedicine services in remote treatment. IEEE KhPI Week on Advanced Technology, KhPI Week, 2020, 347–350.
• [7] GitHub [https://github.com/googlesamples/mediapipe/tree/main/examples/hand_landmarker/ios] (accessed 19 Feb. 2024).
• [8] Google for Developers. Hand landmarks detection guide [https://developers.google.com/mediapipe/solutions/vision/hand_landmarker#model] (accessed 19 Feb. 2024).
• [9] Gupta S., White M.: Improved On-Device ML on Pixel 6, with Neural Architecture Search. Google Research Blog [https://blog.research.google/2021/11/improved-on-device-ml-on-pixel-6-with.html] (accessed 8 Nov. 2021).
• [10] Kim B., Neville Ch.: Accuracy and Feasibility of a Novel Fine Hand Motor Skill Assessment Using Computer Vision Object Tracking. Scientific Reports 13(1), 2023, 1–14 [https://doi.org/10.1038/s41598-023-29091-0].
• [11] Lin T.-Y. et al.: Feature Pyramid Networks for Object Detection [https://arxiv.org/pdf/1612.03144.pdf].
• [12] Liu W. et al.: SSD: Single Shot MultiBox Detector [https://arxiv.org/pdf/1512.02325.pdf].
• [13] Liang M. et al.: Deep Continuous Fusion for Multi-Sensor 3D Object Detection. 2020 [https://arxiv.org/abs/2012.10992] (accessed 19 Feb. 2024).
• [14] Liang M. et al.: Multi-Task Multi-Sensor Fusion for 3D Object Detection [https://openaccess.thecvf.com/content_CVPR_2019/papers/Liang_MultiTask_ MultiSensor_Fusion_for_3D_Object_Detection_CVPR_2019_paper.pdf].
• [15] Muhammad B. S., Chai D.: RGB-D Data-Based Action Recognition: A Review. Sensors 21(12), 2021, 4246–4246 [https://doi.org/10.3390/s21124246].
• [16] On-Device, Real-Time Hand Tracking with MediaPipe [https://blog.research.google/2019/08/on-device-real-time-hand-trackingwith.html] (accessed 19 Feb. 2024).
• [17] Romanyuk O. et al.: A function-based approach to real-time visualization using graphics processing units. Proc. SPIE 11581, 2020, 115810E [https://doi.org/10.1117/12.2580212].
• [18] Selivanova K. Avrunin O.: Method of Hand Movement Disorders Determination based on the Surgeon's Laparoscopic Video Recording. 3rd KhPI Week on Advanced Technology – KhPIWeek, 2022, 1–4 [https://doi.org/10.1109/KhPIWeek57572.2022.9916457].
• [19] Selivanova K. et al.: The tracking system of a three-dimensional position of hand movement for tremor detection. Proc. SPIE 11581, 2020, 115810I [https://doi.org/10.1117/12.2580330].
• [20] Sokol Y. et al.: Using medical imaging in disaster medicine. IEEE 4th International Conference on Intelligent Energy and Power Systems, IEPS 2020, 2020, 287–290.
• [21] Taeger J. et al.: Utilization of Smartphone Depth Mapping Cameras for App-Based Grading of Facial Movement Disorders: Development and Feasibility Study. JMIR mHealth and uHealth 9(1), 2021, e19346 [https://doi.org/10.2196/19346].
• [22] Turlykozhayeva D. et al.: Routing Algorithm for Software Defined Network Based on Boxcovering Algorithm. 10th International Conference on Wireless Networks and Mobile Communications – WINCOM, 2023.
• [23] Tymkovych M. et al.: 3D scanning technologies by optical RealSense cameras for SIREN-based 3D hand representation. Proc. SPIE 12985, 2023, 129850O [https://doi.org/10.1117/12.3022737].
• [24] Urban S. et al.: On the Issues of TrueDepth Sensor Data for Computer Vision Tasks Across Different IPad Generations. 2022 [https://arxiv.org/abs/2201.10865] (accessed 26 Nov. 2023).
• [25] WójcikW. et al.: Information Technology in Medical Diagnostics II. Taylor & Francis Group. CRC Press, Balkema Book. London, 2019.
• [26] Wójcik W. et al.: Information Technology in Medical Diagnostics. CRC Press, 2017.
• [27] Zhang F. et al.: MediaPipe Hands: On-device Real-time Hand Tracking. 2006 [https://arxiv.org/abs/2006.10214]