A universal architectural pattern and specification method for robot control system design

• Autorzy: Kornuta T. , Zieliński C. , Winiarski T.

Identyfikatory

DOI

10.24425/bpasts.2020.131827

• Języki publikacji: EN

Abstrakty

EN

The paper presents a universal architectural pattern and an associated specification method that can be applied in the design of robot control systems. The approach a describes the system in terms of embodied agents and proposes a multi-step decomposition enabling precise definition of their inner structure and operation. An embodied agent is decomposed into effectors, receptors, both real and virtual, and a control subsystem. Those entities communicate through communication buffers. The activities of those entities are governed by FSMs that invoke behaviours formulated in terms of transition functions taking as arguments the contents of input buffers and producing the values inserted into output buffers. The method is exemplified by applying it to the design of a control system of a robot executing one of the most important tasks for a service robot, i.e. picking up, by a position-force controlled robot, an object located using an RGB-D image acquired from a Kinect. Moreover in order to substantiate the universality of the presented approach we present how classical, known from the literature, robotic architectures can be expressed as systems composed of one or more embodied agents.

Słowa kluczowe

EN

autonomous agents; control architectures and programming; service robots; range sensing; recognition; grasping

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2020
• Tom: Vol. 68, nr 1
• Strony: 3--29
• Opis fizyczny: Bibliogr. 93 poz., rys., tab.

Twórcy

autor

Kornuta T.

• IBM Research, Almaden Research Center
• Warsaw University of Technology, Institute of Control and Computation Engineering

autor

Zieliński C.

• Warsaw University of Technology, Institute of Control and Computation Engineering

autor

Winiarski T.

• Warsaw University of Technology, Institute of Control and Computation Engineering

Bibliografia

• [1] R. Alami, R. Chatila, S. Fleury, M.M. Ghallab, and F. Ingrand, “An architecture for autonomy”, Int. J. of Robotics Research 17 (4), 315–337 (1998).
• [2] A. Aldoma, F. Tombari, R.B. Rusu, and M. Vincze, OUR-CVFH–Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6DOF Pose Estimation, Springer, 2012.
• [3] S. Ambroszkiewicz, W. Bartyna, M. Faderewski, and G. Terlikowski, “Multirobot system architecture: environment representation and protocols”, Bull. Pol. Ac.: Tech. 58 (1), 3–13 (2010).
• [4] R. C. Arkin, Behavior-Based Robotics, MIT Press, 1998.
• [5] C. Armbrust, L. Kiekbusch, T. Ropertz, and K. Berns, Soft robot control with a behaviour-based architecture, In Soft Robotics, pages 81–91. Springer, 2015.
• [6] V. Azizi, A. Kimmel, K. Bekris, and M. Kapadia, Geometric reachability analysis for grasp planning in cluttered scenes for varying end-effectors, In Automation Science and Engineering (CASE), 2017 13th IEEE Conference on, pages 764–769. IEEE, 2017.
• [7] M. Beetz, L. Mösenlechner, and M. Tenorth, CRAMm – a cognitive robot abstract machine for everyday manipulation in human environments, In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, October 18–22, 2010, Taipei, Taiwan, pages 1012–1017. IEEE, 2010.
• [8] M. O. Ben Salem, O. Mosbahi, M. Khalgui, Z. Jlalia, G. Frey, and M. Smida, “Brometh: Methodology to design safe reconfigurable medical robotic systems”, The International Journal of Medical Robotics and Computer Assisted Surgery 13 (3), e1786 (2017).
• [9] S. Bensalem, L. de Silva, F. Ingrand, and R. Yan, “A verifiable and correct-by-construction controller for robot functional levels”, Journal of Software Engineering for Robotics, 2 (1), 1–19 (2011).
• [10] Ł. Białek, M. Borkowska, A. Borkowski, B. Dunin-Kęplicz, M. Gnatowski, A. Szałas, and J. Szklarski, “Coordinating multiple rescue robots”, Prace Naukowe Politechniki Warszawskiej. Elektronika, (194), 185–196 (2014).
• [11] F. Bonsignorio, “A new kind of article for reproducible research in intelligent robotics [from the field]”, IEEE Robotics & Automation Magazine 24 (3), 178–182 (2017).
• [12] G. Bradski and A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, O’Reilly, first edition, September 2008.
• [13] R.A. Brooks, “A robust layered control system for a mobile robot”, IEEE Journal of Robotics and Automation 2 (1), 14–23 (1986).
• [14] R.A. Brooks, “Intelligence without representation”, Artificial Intelligence 47 (1-3), 139–159, January (1991).
• [15] D. Brugali, “Model-driven software engineering in robotics”, IEEE Robotics Automation Magazine 22 (3), 155–166, Sept (2015).
• [16] H. Bruyninckx, The real-time motion control core of the OROCOS project, In Proceedings of the IEEE International Conference on Robotics and Automation, pages 2766–2771, IEEE, September 2003.
• [17] H. Bruyninckx and J. De Schutter, “Specification of Force-Controlled Actions in the Task Frame Formalism: A Synthesis”, IEEE Trans. on Robotics and Automation 12 (4), 581–589, August (1996).
• [18] E. Cervera, “Try to start it! the challenge of reusing code in robotics research”, IEEE Robotics and Automation Letters 4 (1), 49–56 (2019).
• [19] H. Chen and B. Bhanu, “3d free-form object recognition in range images using local surface patches”, Pattern Recognition Letters 28 (10), 1252–1262 (2007).
• [20] D. Choiński and M. Senik, “Distributed control systems integration and management with an ontology-based multi-agent system”, Bull. Pol. Ac.: Tech. 66 (5), 613–620 (2018).
• [21] A. Collet, M. Martinez, and S.S. Srinivasa, “The MOPED framework: Object Recognition and Pose Estimation for Manipulation”, The International Journal of Robotics Research 30 (10), 1284–1306 (2011).
• [22] S. Coradeschi and A. Saffiotti, “An introduction to the anchoring problem”, Robotics and Autonomous Systems 43 (2), 85–96 (2003).
• [23] C. Crick, G. Jay, S. Osentoski, B. Pitzer, and O.C. Jenkins, Rosbridge: Ros for non-ros users, In Robotics Research, pages 493–504, Springer, 2017.
• [24] W. Dudek, W. Szynkiewicz, and T. Winiarski, “Cloud computing support for the multi-agent robot navigation system”, Journal of Automation Mobile Robotics and Intelligent Systems 11 (2), 67–74 (2017).
• [25] M. Figat, C. Zieliński, and R. Hexel, Fsm based specification of robot control system activities, In 11th International Workshop on Robot Motion and Control (RoMoCo), pages 193–198, July 2017.
• [26] S. Fleury, M. Herrb, and R. Chatila, “GenoM: A tool for the specification and the implementation of operating modules in a distributed robot architecture”, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’97), 2, 842–849, September (1997).
• [27] E. Gamma, J. Vlissides, R. Johnson, and R. Helm, Design patterns: elements of reusable object-oriented software, Addison-Wesley, 1994.
• [28] E. Gat, On three-layer architectures, In D. Kortenkamp, R.P. Bonnasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots, pages 195–210, AAAI Press Cambridge, MA, 1998.
• [29] H. Golnabi and A. Asadpour, “Design and application of industrial machine vision systems” Robotics and Computer-Integrated Manufacturing 23 (6), 630–637 (2007).
• [30] A.R. Graves and C. Czarnecki, “Design patterns for behavior-based robotics”, IEEE Transactions on Systems, Man, and Cybernetics – part A: Systems and Humans 30 (1), 36–41 (2000).
• [31] S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, and V. Lepetit, Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes, In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 858–865, IEEE, 2011.
• [32] T. Hodan, F. Michel, E. Brachmann, W. Kehl, A. GlentBuch, D. Kraft, B. Drost, J. Vidal, S. Ihrke, et al., Bop: benchmark for 6d object pose estimation, In Proceedings of the European Conference on Computer Vision (ECCV), pages 19–34, 2018.
• [33] M. Hutson, “Artificial intelligence faces reproducibility crisis”, Science 359 (6377), 725–726 (2018).
• [34] M. Janiak and C. Zieliński, “Control system architecture for the investigation of motion control algorithms on an example of the mobile platform Rex”, Bull. Pol. Ac.: Tech. 63 (3), 667–678 (2015).
• [35] J. Winkler, M. Tenorth, A.K. Bozcuoglu, and M. Beetz, CRAMm – memories for robots performing everyday manipulation activities, In 2nd Annual Conference on Advances in Cognitive Systems, pages 91–108, 2013.
• [36] S. Kaisler, Software Paradigms, Wiley Interscience, 2005.
• [37] W. Kehl, F. Manhardt, F. Tombari, S. Ilic, and N. Navab, Ssd6d: Making rgb-based 3d detection and 6d pose estimation great again, In Proceedings of the International Conference on Computer Vision (ICCV 2017), Venice, Italy, pages 22–29, 2017.
• [38] O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation”, IEEE Journal on Robotics and Automation 3 (1), 43–53 (1987).
• [39] M. Kopicki, R. Detry, M. Adjigble, R. Stolkin, A. Leonardis, and J.L. Wyatt, “One-shot learning and generation of dexterous grasps for novel objects”, The International Journal of Robotics Research, 2015.
• [40] T. Kornuta and M. Stefańczyk, “Modreg: a modular framework for rgb-d image acquisition and 3d object model registration”, Foundations of Computing and Decision Sciences 42 (3), 183–201 (2017).
• [41] T. Kornuta, T. Winiarski, and C. Zieliński, Specification of abstract robot skills in terms of control system behaviours, In R. Szewczyk, C. Zieliński, and M. Kaliczyńska, editors, Progress in Automation, Robotics and Measuring Techniques. Vol. 2. Robotics, volume 351 of Advances in Intelligent Systems and Computing (AISC), pages 139–152, Springer, 2015.
• [42] T. Kornuta and C. Zieliński, “Robot control system design exemplified by multi-camera visual servoing”, Journal of Intelligent & Robotic Systems 77 (3–4), 499–524 (2015).
• [43] D. Kortenkamp and R. Simmons, Robotic systems architectures and programming, In O. Khatib and B. Siciliano, editors, Springer Handbook of Robotics, pages 187–206, Springer, 2008.
• [44] D. Kortenkamp, R. Simmons, and D. Brugali, Robotic systems architectures and programming, In B. Siciliano and O. Khatib, editors, Springer Handbook of Robotics, 2nd Edition, pages 283–306, Springer, 2016.
• [45] A. Koubâa, Robot operating system (ros): The complete reference, volume 2, Springer, 2017.
• [46] R. Laganiere, OpenCV 3 Computer Vision Application Programming Cookbook, Packt Publishing Ltd, 2017.
• [47] M. Lauder, M. Schlereth, S. Rose, and A. Schürr, “Model-driven systems engineering: state-of-the-art and research challenges”, Bull. Pol. Ac.: Tech. 58 (3), 409–421 (2010).
• [48] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A.C. Berg, Ssd: Single shot multibox detector, In European conference on computer vision, pages 21–37, Springer, 2016.
• [49] D.G. Lowe, “Distinctive image features from scale-invariant keypoints”, International Journal of Computer Vision 60 (2), 91–110 (2004).
• [50] R. Lumia, J. Fiala, and A. Wavering, “The nasrem robot control system standard”, Robotics and Computer-Integrated Manufacturing 6 (4), 303–308 (1989). Special Issue Robots in Manufacturing.
• [51] D.M. Lyons, Prerational intelligence, volume 2: Adaptive behavior and intelligent systems without symbols and logic of Studies in cognitive systems, chapter A Schema-Theory Approach to Specifying and Analysing the Behavior of Robotic Systems, pages 51–70, Kluwer Academic, 2001.
• [52] D.M. Lyons and M.A. Arbib, “A formal model of computation for sensory-based robotics”, IEEE Transactions on Robotics and Automation 5 (3), 280–293, June (1989).
• [53] D.M. Lyons, R.C. Arkin, S. Jiang, M. O’Brien, F. Tang, and P. Tang, “Performance verification for robot missions in uncertain environments”, Robotics and Autonomous Systems 98, 89–104 (2017).
• [54] M. Mason, “Compliance and force control for computer controlled manipulators”, IEEE Transactions on Systems, Man, and Cybernetics (11), 418–432 (1981).
• [55] A.T. Miller and P.K. Allen, “Graspit! a versatile simulator for robotic grasping”, Robotics & Automation Magazine, IEEE 11 (4), 110–122 (2004).
• [56] M. Muja and D.G. Lowe, Fast approximate nearest neighbors with automatic algorithm configuration, In VISAPP (1), pages 331–340, 2009.
• [57] S. Müller, P. Wolf, K. Berns, and P. Liggesmeyer, Combining behavior-based and contract-based control architectures for behavior optimization of networked autonomous vehicles in unstructured environments, In Commercial Vehicle Technology 2018, pages 324–335, Springer, 2018.
• [58] I. Nesnas, The CLARAty project: Coping with hardware and software heterogenity, In D. Brugali, editor, Software Engineering for Experimental Robotics, pages 9–30, Springer–Verlag, 2007.
• [59] D. Pangercic, M. Tenorth, D. Jain, and M. Beetz, Combining perception and knowledge processing for everyday manipulation, In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan, pages 1065–1071, 2010.
• [60] C.R. Qi, H. Su, K. Mo, and L.J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation”, Proc. Computer Vision and Pattern Recognition (CVPR), IEEE 1 (2), 4 (2017).
• [61] R.B. Rusu and S. Cousins, 3D is here: Point Cloud Library (PCL), In International Conference on Robotics and Automation, Shanghai, China, 2011.
• [62] S. Sabour, N. Frosst, and G.E. Hinton, Dynamic routing between capsules, In Advances in Neural Information Processing Systems, pages 3856–3866, 2017.
• [63] B. Sellner, F.W. Heger, L.M. Hiatt, R. Simmons, and S. Singh, “Coordinated multiagent teams and sliding autonomy for large-scale assembly”, Proceedings of the IEEE – Special Issue on Multi-Robot Systems 94 (7), 1425–1444, July (2006).
• [64] D. Seredyński and W. Szynkiewicz, Fast Grasp Learning for Novel Objects, In R. Szewczyk, C. Zieliński, and M. Kaliczyńska, editors, Recent Advances in Automation, Robotics and Measuring Techniques, volume 440 of Advances in Intelligent Systems and Computing (AISC), pages 681–692. Springer, 2016.
• [65] D. Seredyński, T. Winiarski, K. Banachowicz, and C. Zieliński, Grasp planning taking into account the external wrenches acting on the grasped object, In Robot Motion and Control (RoMoCo), 10th International Workshop on, pages 40–45. IEEE, 2015.
• [66] B.R. Shetty and M.H. Ang, Active compliance control of a puma 560 robot, In Robotics and Automation, 1996. Proceedings., 1996 IEEE International Conference on, volume 4, pages 3720–3725, IEEE, 1996.
• [67] K. Shoemake, Animating rotation with quaternion curves, In ACM SIGGRAPH computer graphics, volume 19, pages 245–254, ACM, 1985.
• [68] P. Skrzypczyński, “Multi-agent software architecture for autonomous robots: A practical approach”, Management and Production Engineering Review 1 (4), 55–66, December (2010).
• [69] M. Staniak, T. Winiarski, and C. Zieliński, Parallel visual-force control, In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2008.
• [70] M. Staniak and C. Zieliński, “Structures of visual servos”, Robotics and Autonomous Systems 58 (8), 940–954 (2010).
• [71] M. Stefańczyk and T. Kornuta, Handling of asynchronous data flow in robot perception subsystems, In Simulation, Modeling, and Programming for Autonomous Robots, volume 8810 of Lecture Notes in Computer Science, pages 509–520, Springer, 2014.
• [72] M. Stefańczyk, M. Laszkowski, and T. Kornuta, WUT Visual Perception Dataset – a dataset for registration and recognition of objects, In Challenges in Automation, Robotics and Measurement Techniques, volume 440 (2) of Advances in Intelligent Systems and Computing (AISC), pages 635–645, Springer, 2016.
• [73] M. Stenmark and J. Malec, “Knowledge-Based Instruction of Manipulation Tasks for Industrial Robotics”, Robotics and Computer Integrated Manufacturing, 33, 56–67, June (2015).
• [74] M. Stenmark, J. Malec, K. Nilsson, and A. Robertsson, “On distributed knowledge bases for robotized small-batch assembly”, IEEE Transactions on Automation Science and Engineering 12 (2), 519–528 (2015).
• [75] C. Szegedy, S. Ioffe, V. Vanhoucke, and A.A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, In AAAI, volume 4, page 12, 2017.
• [76] M. Tenorth and M. Beetz, “KnowRob: a knowledge processing infrastructure for cognition-enabled robots”, International Journal of Robotics Research 32 (5), 566–590, May (2013).
• [77] M. Walęcki, K. Banachowicz, and T. Winiarski, Research oriented motor controllers for robotic applications, In K. Kozłowski, editor, Robot Motion and Control 2011 (LNCiS) Lecture Notes in Control & Information Sciences, volume 422, pages 193–203, Springer Verlag London Limited, 2012.
• [78] T. Winiarski and K. Banachowicz, Automated generation of component system for the calibration of the service robot kinematic parameters, In 20th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR, pages 1098–1103, IEEE, 2015.
• [79] T. Winiarski, K. Banachowicz, M. Walęcki, and J. Bohren, Multibehavioral position-force manipulator controller, In 21th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR, pages 651–656, IEEE, 2016.
• [80] T. Winiarski and A. Woźniak, “Indirect force control development procedure”, Robotica 31 (3), 465–478 (2013).
• [81] Y. Zhao, T. Birdal, H. Deng, and F. Tombari, “3d point-capsule networks”, arXiv preprint arXiv:1812.10775, 2018.
• [82] C. Zieliński, The MRROC++ system, In Proceedings of the First Workshop on Robot Motion and Control, RoMoCo’99, pages 147–152, June 1999.
• [83] C. Zieliński, A Quasi-Formal Approach to Structuring Multi-Robot System Controllers, In Second International Workshop on Robot Motion and Control, RoMoCo’01, pages 121–128, 18–20 October 2001.
• [84] C. Zieliński, “Formal approach to the design of robot programming frameworks: the behavioural control case” Bull. Pol. Ac.: Tech. 53 (1), 57–67, March (2005).
• [85] C. Zieliński, M. Figat, and R. Hexel, “Communication within multi-fsm based robotic systems”, Journal of Intelligent & Robotic Systems 93 (3-4), 787–805 (2019).
• [86] C. Zieliński and T. Kornuta, Generation of linear Cartesian trajectories for robots using industrial motion-controllers, In 16th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR, pages 62–67, August 2011.
• [87] C. Zieliński and T. Kornuta, Diagnostic requirements in multi-robot systems, In Intelligent Systems in Technical and Medical Diagnostics, volume 230 of Advances in Intelligent Systems and Computing (AISC), pages 345–356, Springer, 2014.
• [88] C. Zieliński, T. Kornuta, and T. Winiarski, A systematic method of designing control systems for service and field robots, In 19th IEEE International Conference on Methods and Models in Automation and Robotics, MMAR, pages 1–14. IEEE, 2014.
• [89] C. Zieliński, M. Stefańczyk, T. Kornuta, M. Figat, W. Dudek, W. Szynkiewicz, W. Kasprzak, J. Figat, M. Szlenk, T. Winiarski, K. Banachowicz, T. Zielińska, E. G. Tsardoulias, A. L. Symeonidis, F.E. Psomopoulos, A.M. Kintsakis, P.A. Mitkas, A. Thallas, S.E. Reppou, G.T. Karagiannis, K. Panayiotou, V. Prunet, M. Serrano, J.-P. Merlet, S. Arampatzis, A. Giokas, L. Penteridis, I. Trochidis, D. Daney, and M. Iturburu, “Variable structure robot control systems: The RAPP approach”, Robotics and Autonomous Systems 94, 226–244 (2017).
• [90] C. Zieliński, W. Szynkiewicz, T. Winiarski, M. Staniak, W. Czajewski, and T. Kornuta, “Rubik’s cube as a benchmark validating MRROC++ as an implementation tool for service robot control systems”, Industrial Robot: An International Journal 34 (5), 368–375 (2007).
• [91] C. Zieliński and T. Winiarski, “General specification of multi-robot control system structures”, Bull. Pol. Ac.: Tech. 58 (1), 15–28 (2010).
• [92] C. Zieliński and T. Winiarski, “Motion generation in the MRROC++ robot programming framework”, International Journal of Robotics Research 29 (4), 386–413 (2010).
• [93] C. Zieliński, T. Winiarski, and T. Kornuta, Agent-based structures of robot systems, In J. Kacprzyk and et al, editors, Trends in Advanced Intelligent Control, Optimization and Automation, volume 577 of Advances in Intelligent Systems and Computing, pages 493–502, 2017.

Uwagi

PL

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2020).