Human motion capture using Kinect and third party sensors

• Autorzy: Żorski W. , Pałys T.

Warianty tytułu

PL

Przechwytywanie ruchów i gestów za pomocą Kinecta i specjalizowanych sensorów

• Języki publikacji: EN

Abstrakty

EN

The paper describes an applicable solution combining advantages of the Kinect device and properties of appropriable third party sensors. The elaborated solution allows tracking of a human arm along with recognition of basic hand gestures. This way it may be possible to remotely control a manipulator or a robotic arm performing some actions determined by a user's hand. The Kinect is mainly used to preliminary calibrate the system and for verification purposes. The system was designed using kinematics-based approach with rigid transformation combining rotations and translations. Matrix transformation operators were exchanged by dual quaternions as quaternions are native for the used devices. Additionally, as this is not a trivial mathematical tool, the machinery of dual quaternions has been introduced and its implementation is given.

PL

Artykuł przedstawia rozwiązanie problemu śledzenia ręki poruszającego się człowieka, wraz z rozpoznawaniem prostych gestów, przy wykorzystaniu właściwości urządzenia Kinect oraz dodatkowych sensorów. Rozwiązanie może być stosowane do zdalnego sterowania manipulatorem za pomocą ręki i gestów dłoni. Urządzenie Kinect służy głównie na etapie kalibracji systemu oraz w celu weryfikacji jego działania. System został pierwotnie zaprojektowany przy użyciu standardowej kinematyki manipulatora, opartej na macierzowych operatorach przekształcenia, które następnie zostały zastąpione przez kwaterniony dualne, gdyż są one wykorzystywane natywnie przez zastosowane urządzenia. Artykuł zawiera krótkie wprowadzenie do kwaternionów dualnych oraz ich przykładową implementację w środowisku Scilab.

Słowa kluczowe

EN

Kinect; space sensors; robot kinematics; rigid transformation; dual quaternions

PL

Kinect; sensory; kinematyka manipulatora; operator przekształcania; kwaterniony dualne

• Wydawca: Instytut Teleinformatyki i Automatyki, Wydział Cybernetyki, Wojskowa Akademia Techniczna im. Jarosława Dąbrowskiego
• Czasopismo: Przegląd Teleinformatyczny
• Rocznik: 2016
• Tom: T. 4, Nr 1-2 (41)
• Strony: 17--38
• Opis fizyczny: Bibliogr. 26 poz., il., rys., wykr.

Twórcy

autor

Żorski W.

• Instytut Teleinformatyki i Automatyki WAT, ul. Gen. S. Kaliskiego 2, 00-908 Warszawa

autor

Pałys T.

• Instytut Teleinformatyki i Automatyki WAT, ul. Gen. S. Kaliskiego 2, 00-908 Warszawa

Bibliografia

• [1] ABDULLAH M. S. H., ZABIDI A., YASSIN I. M., HASSAN H. A., Analysis of Microsoft Kinect depth perception for distance detection of vehicles. IEEE 6th Control and System Graduate Research Colloquium, ICSGRC 2015, pp. 116-119.
• [2] BENABDALLAH I., BOUTERAA Y., BOUCETTA R., REKIK C., Kinect-based Computed Torque Control for lynxmotion robotic arm. 7th International Conference on Modelling, Identification and Control, ICMIC 2015, pp. 1-6.
• [3] CRAIG J. J., Introduction to Robotics: mechanics and control. Addison-Wesley Publishing Company, Singapore, 1989.
• [4] EL-LAITHY R.A., HUANG J., YEH M., Study on the use of Microsoft Kinect for robotics applications. Position Location and Navigation Symposium (PLANS), 2012, pp. 1280-1288.
• [5] FANKHAUSER P., et al., Kinect v2 for Mobile Robot Navigation: Evaluation and Modeling. IEEE International Conference on Advanced Robotics, ICAR 2015, pp. 388-394.
• [6] GOUASMI M., OUALI M., BRAHIM F., Robot Kinematics Using Dual Quaternions. International Journal of Robotics and Automation (IJRA), Vol. 1, No. 1, March 2012, pp. 13-30.
• [7] HACHAJ T., OGIELA M. R., KOPTYRA K., Effectiveness Comparison of Kinect and Kinect 2 for Recognition of Oyama Karate Techniques. 18th International Conference on Network-Based Information Systems, NBiS, 2015, pp. 332-337.
• [8] HAN J., SHAO L., XU D., SHOTTON J., Enhanced Computer Vision With Microsoft Kinect Sensor: A Review. IEEE Transactions on Cybernetics, Vol. 43, No. 5., 2013, pp. 1318-1334.
• [9] ISLAM M. R., et al., A Novel Approach for Constructing Emulator for Microsoft Kinect XBOX 360 Sensor in the .NET Platform. 4th International Conference on Intelligent Systems, Modeling and Simulation, 2013, pp. 1-6.
• [10] JAIS H. M., MAHAYUDDIN Z. R., ARSHAD H., A review on gesture recognition using Kinect, International Conference on Electrical Engineering and Informatics, ICEEI 2015, pp. 594-599.
• [11] JUNGONG H., LING S, DONG XU, SHOTTON J., Enhanced Computer Vision with Microsoft Kinect Sensor: A Review. IEEE Transactions on Cybernetics, Vol. 43(5), 2013, pp. 1318-1334.
• [12] KAVAN L., COLLINS S., ZARA J., O'SULLIVAN C., Skinning with dual quaternions. Proceedings of the 2007 symposium on Interactive 3D graphics and games, I3D 2007, pp. 39-46.
• [13] KENWRIGHT B., A Beginners Guide to Dual-Quaternions. WSCG 2012 Communication Proceedings, pp.1-13.
• [14] KIM JUNG-HA, KUMAR V. R., Kinematics of robot manipulators via line transformations. Journal of Robotic Systems, Vol. 7(4), 1990, pp. 649–674.
• [15] KUIPERS J. B., Quaternions and Rotation Sequences: A Primer with Applications to Orbits. Aerospace and Virtual Reality, Princeton University Press, 2002.
• [16] MAJDI A., BAKKAY M. C., ZAGROUBA E., 3D modeling of indoor environments using Kinect sensor. Second International Conference on Image Information Processing, ICIIP 2013, pp. 67-72.
• [17] MING A., ENOMOTO K., SHINOZAKI M., SATO R., SHIMOJO M., Development of an entertainment robot system using Kinect. 8th Europe-Asia Congress on Mechatronics, 2014, pp. 127-132.
• [18] PAGLIARI D., MENNA F., RONCELLA R., REMONDINO F., PINTO L., Kinect Fusion improvement using depth camera calibration. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5, 2014, pp. 479-485.
• [19] PAŁYS T., ŻORSKI W., Enhanced movement tracking with Kinect supported by high-precision sensors. Proceedings of the Federated Conference on Computer Science and Information Systems 2015, pp. 883-888.
• [20] PATSADU O., NUKOOLKIT C. WATANAPA B., Human gesture recognition using Kinect camera. The Proceeding of International Joint Conference on Computer Science and Software Engineering, JCSSE 2012, pp. 28-32.
• [21] REN YU, GUBING LU, FENG LU, A Method of Rotation Transformation for 3D Object by Changing Camera Attributes in WPF. International Conference on Information Engineering and Computer, 2010, pp. 1-4.
• [22] SHIRWALKAR S., SINGH A., SHARMA K., SINGH N., Telemanipulation of an industrial robotic arm using gesture recognition with Kinect. International Conference on Control, Automation, Robotics and Embedded Systems, CARE 2013, pp. 1-6.
• [23] STONE E. E., et al., Evaluation of the Microsoft Kinect for screening ACL injury. 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2013, pp. 4152-4155.
• [24] WANG T., BU L., HUANG Z., A new method for obstacle detection based on Kinect depth image. Chinese Automation Congress (CAC), 2015, pp. 537-541.
• [25] WU H., et al., Kinect-based robotic manipulation: From human hand to endeffector. IEEE 10th Conference on Industrial Electronics and Applications, ICIEA 2015, pp. 806-811.
• [26] WU L., CHAI S., Depth Filtering in 3D Reconstruction of Indoor Scenes Based on Kinect. Seventh International Symposium on Computational Intelligence and Design, ISCID 2014, pp. 356-359.