ModReg: a modular framework for RGB-D image acquisition and 3D object model registration

• Autorzy: Kornuta T. , Stefańczyk M.

Identyfikatory

DOI

10.1515/fcds-2017-0009

• Języki publikacji: EN

Abstrakty

EN

RGB-D sensors became a standard in robotic applications requiring object recognition, such as object grasping and manipulation. A typical object recognition system relies on matching of features extracted from RGB-D images retrieved from the robot sensors with the features of the object models. In this paper we present ModReg: a system for registration of 3D models of objects. The system consists of a modular software associated with a multi-camera setup supplemented with an additional pattern projector, used for the registration of high-resolution RGB-D images. The objects are placed on a fiducial board with two dot patterns enabling extraction of masks of the placed objects and estimation of their initial poses. The acquired dense point clouds constituting subsequent object views undergo pairwise registration and at the end are optimized with a graph-based technique derived from SLAM. The combination of all those elements resulted in a system able to generate consistent 3D models of objects.

Słowa kluczowe

EN

ModReg; textured stereo; RGB-D image; point cloud; registration

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2017
• Tom: Vol. 42, No. 3
• Strony: 189--201
• Opis fizyczny: Bibliogr. 40 poz., fig., tab.

Twórcy

autor

Kornuta T.

• IBM Research, Almaden, 650 Harry Rd, San Jose, CA 95120

autor

Stefańczyk M.

• Warsaw University of Technology, Institute of Control and Computation Engineering

Bibliografia

• 1. Aldoma A., Tombari F., Di Stefano L., and Vincze M. A global hypotheses verification method for 3D object recognition. In Computer Vision (ECCV 2012), pages 511-524. Springer, 2012.
• 2. Aldoma A., Tombari F., Prankl J., Richtsfeld A., Di Stefano L., and Vincze M. Multimodal cue integration through hypotheses verification for RGB-D object recognition and 6DOF pose estimation. In Robotics and Automation (ICRA), 2013 IEEE International Conference on, pages 2104-2111. IEEE, 2013.
• 3. Belter D., Nowicki M., Skrzypczynski P., Walas K., and Wietrzykowski J. Lightweight RGB-D SLAM System for Search and Rescue Robots. In Progress in Automation, Robotics and Measuring Techniques, pages 11-21. Springer, 2015.
• 4. Besl P. and McKay N. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239-256, 1992.
• 5. Bradski G. and Kaehler A. Learning OpenCV: Computer vision with the OpenCV library. O’Reilly Media, 2008.
• 6. Correll N., Bekris K.E., Berenson D., Brock O., Causo A., Hauser K., Okada K., Rodriguez A., Romano J.M., and Wurman P.R. Analysis and observations from the first amazon picking challenge. IEEE Transactions on Automation Science and Engineering, 2016.
• 7. Dziergwa M., Kaczmarek P., and Kędzierski J. RGB-D Sensors in Social Robotics. Journal of Automation Mobile Robotics and Intelligent Systems, 9(1):18-27, 2015.
• 8. Figat J., Kornuta T., and Kasprzak W. Performance evaluation of binary descriptors of local features. In Chmielewski L., Kozera R., Shin B.-S., and Wojciechowski K., editors, Proceedings of the International Conference on Computer Vision and Graphics, volume 8671 of Lecture Notes in Computer Science, pages 187-194. Springer Berlin / Heidelberg, 2014.
• 9. Firman M. RGB-D datasets: Past, present and future. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 19-31, 2016.
• 10. Großmann B., Siam M., and Krüger V. Comparative evaluation of 3D pose estimation of industrial objects in RGB pointclouds. In Computer Vision Systems, pages 329-342. Springer, 2015.
• 11. Hirschmuller H. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell., pages 328-341, II 2008.
• 12. Holz D., Ichim A.E., Tombari F., Rusu R.B., and Behnke S. Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D. Robotics & Automation Magazine, IEEE, 22(4):110-124, 2015.
• 13. Konolige K. Projected texture stereo. In International Conference on Robotics and Automation (ICRA), pages 148-155. IEEE, 2010.
• 14. Kornuta T. and Laszkowski M. Perception subsystem for object recognition and pose estimation in RGB-D images. In Szewczyk R., Zieliński C., and Kaliczyńska M., editors, Recent Advances in Automation, Robotics and Measuring Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 597-607. Springer, 2016.
• 15. Kornuta T. and Stefańczyk M. Acquisition of RGB-D images: sensors (in Polish). Pomiary - Automatyka - Robotyka PAR, 18(2):92-99, 2014.
• 16. Kornuta T. and Stefańczyk M. Comparison of methods of acquisition of RGB-D images for the purpose of registration of three-dimensional models of objects (in Polish). In XIV Krajowa Konferencja Robotyki - Postępy robotyki, volume 2, pages 357-366, 2016.
• 17. Kornuta T. and Stefańczyk M. Utilization of textured stereovision for registration of 3D models of objects. In 21th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR’2016, pages 1088-10093. IEEE, 2016.
• 18. Lai K., Bo L., Ren X., and Fox D. A large-scale hierarchical multi-view RGB-D object dataset. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pages 1817-1824. IEEE, 2011.
• 19. Lenz I., Knepper R., and Saxena A. DeepMPC: Learning deep latent features for model predictive control. In Proceedings of Robotics: Science and Systems, Rome, Italy, July 2015.
• 20. Lowe D. Object recognition from local scale-invariant features. In Computer Vision, The Proceedings of the Seventh IEEE International Conference on, volume 2, pages 1150-1157. Ieee, 1999.
• 21. Lu F. and Milios E. Globally consistent range scan alignment for environment mapping. Autonomous Robots, 4(4):333-349, 1997.
• 22. Łępicka M., Kornuta T., and Stefańczyk M. Utilization of colour in ICP-based point cloud registration. In Proceedings of the 9th International Conference on Computer Recognition Systems (CORES 2015), volume 403 of Advances in Intelligent Systems and Computing, pages 821-830. Springer, 2016.
• 23. Muja M. and Lowe D.G. Fast approximate nearest neighbors with automatic algorithm configuration. In VISAPP (1), pages 331-340, 2009.
• 24. Newcombe A.J., Richard A and Davison, Izadi S., Kohli P., Hilliges O., Shot- ton J., Molyneaux D., Hodges S., Kim D., and Fitzgibbon A. KinectFusion: Real-time dense surface mapping and tracking. In Mixed and augmented reality (ISMAR), 2011 10th IEEE international symposium on, pages 127-136. IEEE, 2011.
• 25. Peng X., Sun B., Ali K., and Saenko K. Learning deep object detectors from 3d models. In Proceedings of the IEEE International Conference on Computer Vision, pages 1278-1286, 2015.
• 26. Pomerleau F., Colas F., and Siegwart R. A review of point cloud registration algorithms for mobile robotics. Foundations and Trends in Robotics (FnTROB), 4(1):1-104, 2015.
• 27. Ramey A., Gonzalez-Pacheco V., and Salichs M.A. Integration of a low-cost RGB-D sensor in a social robot for gesture recognition. In Proceedings of the 6th international conference on Human-robot interaction, pages 229-230. ACM, 2011.
• 28. Ren X., Fox D., and Konolige K. Change Their Perception: RGB-D for 3-D Modeling and Recognition. Robotics & Automation Magazine, IEEE, 20(4):49-59, 2013.
• 29. Rusu R., Blodow N., and Beetz M. Fast point feature histograms (FPFH) for 3D registration. In Robotics and Automation, 2009. ICRA ’09. IEEE International Conference on, pages 3212-3217. IEEE, 2009.
• 30. Rusu R.B. and Cousins S. 3D is here: Point Cloud Library (PCL). In International Conference on Robotics and Automation, Shanghai, China, 2011 2011.
• 31. Seredynski D. and Szynkiewicz W. Fast grasp learning for novel objects. In Recent Advances in Automation, Robotics and Measuring Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 681-692. Springer, 2016.
• 32. Sprickerhof J., Nüchter A., Lingemann K., and Hertzberg J. A heuristic loop closing technique for large-scale 6D SLAM. Automatika: Journal for Control, Measurement, Electronics, Computing & Communications, 52(3), 2011.
• 33. Stefańczyk M. and Kornuta T. Acquisition of RGB-D images: methods (in Polish). Pomiary - Automatyka - Robotyka PAR, 18(1):82-90, 2014.
• 34. Stefańczyk M. and Kornuta T. Handling of asynchronous data flow in robot perception subsystems. In Simulation, Modeling, and Programming for Autonomous Robots, volume 8810 of Lecture Notes in Computer Science, pages 509-520. Springer, 2014.
• 35. Stefańczyk M., Laszkowski M., and Kornuta T. WUT Visual Perception Dataset - a dataset for registration and recognition of objects. In Challenges in Automation, Robotics and Measurement Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 635-645. Springer, 2016.
• 36. Sturm J., Engelhard N., Endres F., Burgard W., and Cremers D. A benchmark for the evaluation of RGB-D SLAM systems. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 573-580. IEEE, 2012.
• 37. Tang J., Miller S., Singh A., and Abbeel P. A textured object recognition pipeline for color and depth image data. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 3467-3474. IEEE, 2012.
• 38. Thrun S., Burgard W., and Fox D. A probabilistic approach to concurrent mapping and localization for mobile robots. Autonomous Robots, 5(3-4):253-271, 1998.
• 39. Tombari F. and Di Stefano L. Object recognition in 3D scenes with occlusions and clutter by hough voting. In Image and Video Technology (PSIVT), 2010 Fourth Pacific-Rim Symposium on, pages 349-355. IEEE, 2010.
• 40. Tombari F., Salti S., and Di Stefano L. Unique signatures of histograms for local surface description. In Computer Vision-ECCV 2010, pages 356-369. Springer, 2010.

Uwagi

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).

This is an extended version of the paper presented at the 14th National Conference on Robotics (KKR 2016), Polanica Zdroj, Poland, September 14-18, 2016