Recognition of Hand Posture Based on a Point Cloud Descriptor and a Feature of Extended Fingers

• Autorzy: Warchoł D. , Wysocki M.

Identyfikatory

DOI

10.14313/JAMRIS_1-2016/7

• Języki publikacji: EN

Abstrakty

EN

Our work involves hand posture recognition based on 3D data acquired by the KinectTM sensor in the form of point clouds. We combine a descriptor built on the basis of the Viewpoint Feature Histogram (VFH) with additional feature describing the number of extended fingers. First, we extract a region corresponding to the hand and then a histogram of the edge distances from the palm center is built. Based on quantized version of the histogram we calculate the number of extended fingers. This information is used as a first feature describing the hand which, together with VFH-based features, form the feature vector. Before calculating VFH we rotate the hand making our method invariant to hand rotations around the axis perpendicular to the camera lens. Finally, we apply nearest neighbor technique for the posture classification. We present results of crossvalidation tests performed on a representative dataset consisting of 10 different postures, each shown 10 times by 10 subjects. The comparison of recognition rate and mean computation time with other works performed on this dataset confirms the usefulness of our approach.

Słowa kluczowe

EN

hand posture recognition; depth cameras; Kinect; point cloud; Viewpoint Feature Histogram

• Wydawca: Łukasiewicz Industrial Research Institute for Automation and Measurements PIAP
• Czasopismo: Journal of Automation Mobile Robotics and Intelligent Systems
• Rocznik: 2016
• Tom: Vol. 10, No. 1
• Strony: 48--57
• Opis fizyczny: Bibliogr. 21 poz., rys.

Twórcy

autor

Warchoł D.

• Rzeszów University of Technology, Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Wincentego Pola 2, Rzeszów, Poland, 35-959, tel.: [phone]

autor

Wysocki M.

• Rzeszów University of Technology, Department of Computer and Control Engineering, Faculty of Electrical and Computer Engineering, Wincentego Pola 2, Rzeszów, Poland, 35-959, tel.: [phone]

Bibliografia

• [1] N. Dalal, B. Triggs, “Histograms of oriented gradients for human detection”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego: IEEE, 2005, 886–893. DOI: 10.1109/CVPR.2005.177.
• [2] F. Dominio, M. Donadeo, P. Zanuttigh, “Combining multiple depth-based descriptors for hand gesture recognition”, Pattern Recognition Letters, vol. 50, 2014, pp. 101–111. DOI: 10.1016/j.patrec.2013.10.010.
• [3] F. Jiang, C. Wang, Y. Gao, S. Wu, D. Zhao “Discriminating features learning in hand gesture classification”, IET Computer Vision, vol. 9, 2015, no. 5, 673–680. DOI: 10.1049/iet-cvi.2014.0426.
• [4] P. Garg, N. Aggarwal, S. Sofat, “Vision based hand gesture recognition”, World Academy of Science, Engineering and Technology, vol. 49, 2009, no. 1, 972–977.
• [5] T. Kapuściński, M. Oszust, M. Wysocki, “Recognition of signed dynamic expressions observed by ToF camera”, Signal Processing: Algorithms, Journal of Automation, Mobile Robotics & Intelligent Systems Architectures, Arrangements, and Applications (SPA), Poznań, 2013, 291–296.
• [6] T. Kapuściński, M. Oszust, M. Wysocki, D. Warchoł, “Recognition of hand gestures observed by depth cameras”, International Journal of Advanced Robotic Systems, 2015. DOI: 10.5772/60091.
• [7] C. Keskin, F. Kirac, Y.E. Kara, L. Akarun, “Real time hand pose estimation using depth sensors”. In: IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, 2011, 1228–1234. DOI: 10.1109/ICCVW.2011.6130391.
• [8] G. Koszowski, “Two-pass algorithm for hand pose gesture recognition”. In: XIX National Conference of Discrete Processes Automation, Zakopane, Poland, 2014.
• [9] Y. Li, “Hand gesture recognition using Kinect”, Software Engineering and Service Science (ICSESS), Beijing, 2012, 196–199. DOI: 10.1109/ICSESS.2012.6269439.
• [10] G. Marin, F. Dominio, P. Zanuttigh, “Hand gesture recognition with Leap Motion and Kinect devices”. In: IEEE International Conference on Image Processing (ICIP), Paris, 2014, 1565-1569. DOI: 10.1109/ICIP.2014.7025313.
• [11] J. Molina, A. Escudero-Viñolo, A. Signoriello, M. Pardàs, C. Ferràn, J. Bescós, F. Marqués, J. M. Martínez, “Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models”, Machine Vision and Applications, vol. 24, 2013, no. 1, 187–204. DOI: 10.1007/s00138-011-0364-6.
• [12] S. R. Oprisescu, “Automatic static hand gesture recognition using ToF cameras”. In: European Signal Processing Conference, Bucharest, 2012, 2748–2751.
• [13] Z. Ren, J. Junsong, Y. Meng, Z. Zhang, “Robust part-based hand gesture recognition using Kinect sensor”, IEEE Transactions on Multimedia, vol. 15, 2013, no. 5, 1110–1120. DOI: 10.1109/TMM.2013.2246148.
• [14] Z. Ren, J. Yuan, Z. Zhang, “Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera”. In: 19th ACM International Conference on Multimedia, Scottsdale, 2011, 1093–1096.
• [15] R. B. Rusu, N. Blodow, M. Beetz, “Fast point feature histograms (FPFH) for 3D registration”. In: IEEE Conference on Robotics and Automation, Kobe, 2009, 3212–3217. DOI: 10.1109/ROBOT. 2009.5152473.
• [16] R. B. Rusu, G. Bradski, R. Thibaux, “Fast 3D recognition and pose using the Viewpoint Feature Histogram”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, 2010, 2155–2162. DOI: 10.1109/IROS.2010.5651280.
• [17] R. B. Rusu, S. Cousins, “3D is here: Point Cloud Library (PCL)”. In: IEEE International Conference, Shanghai, 2011, 1–4. DOI: 10.1109/ICRA.2011. 5980567.
• [18] R. Rusu, Z. C. Marton, N. Blodow, “Learning informative point classes for the acquisition of object model maps”. In: Conference on Control, Automation, Robotics and Vision, Hanoi, 2008, 643-650.DOI: 10.1109/ICARCV.2008.4795593.
• [19] D. Uebersax, J. Gall, M. Van den Bergh, “Real-time sign language letter and word recognition from depth data”. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, 2011, 383–390. DOI: 10.1109/ICCVW. 2011.6130267.
• [20] M. Van den Bergh, L. Van Gool, “Combining RGB and ToF cameras for real-time 3D hand gesture interaction”. In: IEEE Workshop on Applications of Computer Vision (WACV), Kona, 2011, 66-72. DOI: 10.1109/WACV.2011.5711485.
• [21] Y. Wen, H. Chuanyah, Y. Guanghui, W. Changbo, “A robust method of detecting hand gestures using depth sensors”. In: IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE), Munich, 2012, 72–77. DOI: 10.1109/HAVE.2012.6374441

Uwagi

PL

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę.