Three-dimensional path-following control of an autonomous underwater vehicle based on deep reinforcement learning

• Autorzy: Liang Zhenyu , Qu Xingru , Zhang Zhao , Chen Cong

Identyfikatory

DOI

10.2478/pomr-2022-0042

• Języki publikacji: EN

Abstrakty

EN

In this article, a deep reinforcement learning based three-dimensional path following control approach is proposed for an underactuated autonomous underwater vehicle (AUV). To be specific, kinematic control laws are employed by using the three-dimensional line-of-sight guidance and dynamic control laws are employed by using the twin delayed deep deterministic policy gradient algorithm (TD3), contributing to the surge velocity, pitch angle and heading angle control of an underactuated AUV. In order to solve the chattering of controllers, the action filter and the punishment function are built respectively, which can make control signals stable. Simulations are carried out to evaluate the performance of the proposed control approach. And results show that the AUV can complete the control mission successfully.

Słowa kluczowe

EN

autonomous underwater vehicle (AUV); three-dimensional path following; deep reinforcement learning-based control; lineof-sight guidance; controller chattering

• Wydawca: Gdańsk University of Technology, Faculty of Ocean Engineering & Ship Technology
• Czasopismo: Polish Maritime Research
• Rocznik: 2022
• Tom: nr 4
• Strony: 36--44
• Opis fizyczny: Bibliogr. 24 poz., rys., tab.

Twórcy

autor

Liang Zhenyu

• School of Mechanical and Electronic Engineering Dalian Minzu University Dalian, 116600 China
• School of Naval Architecture and Ocean Engineering Dalian Maritime University Dalian, 116026 China

• https://orcid.org/0000-0002-8769-4644

autor

Qu Xingru

• School of Mechanical and Electronic Engineering Dalian Minzu University Dalian, 116600 China

• https://orcid.org/0000-0001-5176-333X

autor

Zhang Zhao

• School of Naval Architecture and Ocean Engineering Dalian Maritime University Dalian, 116026 China

autor

Chen Cong

• School of Naval Architecture and Ocean Engineering Dalian Maritime University Dalian, 116026 China

Bibliografia

• 1. I. Stenius et al., “A System for Autonomous Seaweed Farm Inspection with an Underwater Robot,” Sensors, vol. 22, no. 13, Jul 2022, doi: 10.3390/s22135064.
• 2. L. Rowinski and M. Kaczmarczyk, “Evaluation of Effectiveness of Waterjet Propulsor for a Small Underwater Vehicle,” Polish Marit. Res., vol. 28, no. 4, 2022, doi: 10.2478/pomr-2021-0047.
• 3. H. Choukri and L. Z. Qidan, “Path Following Control of Fully Actuated Autonomous Underwater Vehicle Based on LADRC,” Polish Marit. Res., vol. 25, no. 4, 2018, doi: 10.2478/ pomr-2018-0130.
• 4. L. Li, Z. Pei, J. Jin, and Y. Dai, “Control of Unmanned Surface Vehicle along the Desired Trajectory Using Improved Line of Sight and Estimated Sideslip Angle,” Polish Marit. Res., vol. 28, no. 2, 2021, doi: 10.2478/pomr-2021-0017.
• 5. E. Vidal, N. Palomeras, M. Carreras, “Online 3D Underwater Exploration and Coverage,” in IEEE/OES Autonomous Underwater Vehicle Workshop (AUV), Rectory Univ Porto, Porto, Portugal, 2018.
• 6. J. H. Wan et al., “Motion Control of Autonomous Underwater Vehicle Based on Fractional Calculus Active Disturbance Rejection,” Journal of Marine Science and Engineering, vol. 9, no. 11, Nov 2021, doi: 10.3390/jmse9111306.
• 7. J. J. Zhou, X. Y. Zhao, T. Chen, Z. P. Yan, and Z. W. Yang, “Trajectory Tracking Control of an Underactuated AUV Based on Backstepping Sliding Mode With State Prediction,” IEEE Access, vol. 7, 2019, doi: 10.1109/access.2019.2958360.
• 8. M. P. R. Prasad and A. Swarup, “Model predictive control of an AUV using de-coupled approach,” International Journal of Maritime Engineering, vol. 160, Jan‒Mar 2018, doi: 10.3940/ rina.ijme.2018.a1.459. 9.
• 9. J. H. Wan et al., “Multi-strategy fusion based on sea state codes for AUV motion control,” Ocean Engineering, vol. 248, Mar 2022, doi: 10.1016/j.oceaneng.2022.110600.
• 10. Y. K. Xia, K. Xu, Z. M. Huang, W. J. Wang, G. H. Xu, and Y. Li, “Adaptive energy-efficient tracking control of a X rudder AUV with actuator dynamics and rolling restriction,” Applied Ocean Research, vol. 118, Jan 2022, doi: 10.1016/j. apor.2021.102994.
• 11. K. Fang, H. L. Fang, J. W. Zhang, J. Q. Yao, and J. W. Li, “Neural adaptive output feedback tracking control of underactuated AUVs,” Ocean Engineering, vol. 234, Aug 2021, doi: 10.1016/j. oceaneng.2021.109211.
• 12. J. L. Zhang, X. B. Xiang, Q. Zhang, and W. J. Li, “Neural network-based adaptive trajectory tracking control of underactuated AUVs with unknown asymmetrical actuator saturation and unknown dynamics,” Ocean Engineering, vol. 218, Dec 2020, doi: 10.1016/j.oceaneng.2020.108193.
• 13. H. N. Esfahani and R. Szlapczynski, “Model Predictive Super-Twisting Sliding Mode Control for an Autonomous Surface Vehicle,” Polish Marit. Res., vol. 26, no. 3, 2019, doi: 10.2478/ pomr-2019-0057.
• 14. C. X. Cheng, Q. X. Sha, B. He, and G. L. Li, “Path planning and obstacle avoidance for AUV: A review,” Ocean Engineering, vol. 235, Sep 2021, doi: 10.1016/j.oceaneng.2021.109355.
• 15. S. Brandi, M. Fiorentini, and A. Capozzoli, “Comparison of online and offline deep reinforcement learning with model predictive control for thermal energy management,” Automation in Construction, vol. 135, Mar 2022, doi: 10.1016/j.autcon.2022.104128.
• 16. Y. Fang, Z. W. Huang, J. Y. Pu, and J. S. Zhang, “AUV position tracking and trajectory control based on fast-deployed deep reinforcement learning method,” Ocean Engineering, vol. 245, Feb 2022, doi: 10.1016/j.oceaneng.2021.110452.
• 17. P. Zielinski and U. Markowska-Kaczmar, “3D robotic navigation using a vision-based deep reinforcement learning model,” Applied Soft Computing, vol. 110, Oct 2021, doi: 10.1016/j.asoc.2021.107602.
• 18. D. L. Song, W. H. Gan, P. Yao, W. C. Zang, Z. X. Zhang, and X. Q. Qu, “Guidance and control of autonomous surface underwater vehicles for target tracking in ocean environment by deep reinforcement learning,” Ocean Engineering, vol. 250, Apr 2022, doi: 10.1016/j.oceaneng.2022.110947.
• 19. E. Meyer, H. Robinson, A. Rasheed, and O. San, “Taming an Autonomous Surface Vehicle for Path Following and Collision Avoidance Using Deep Reinforcement Learning,” IEEE Access, vol. 8, 2020, doi: 10.1109/access.2020.2976586.
• 20. A. B. Martinsen and A. M. Lekkas, “Straight-Path Following for Underactuated Marine Vessels using Deep Reinforcement Learning,” in 11th IFAC Conference on Control Applications in Marine Systems, Robotics, and Vehicles (CAMS), Opatija, Croatia, 2018, vol. 51.
• 21. T. I. Fossen, Handbook of Marine Craft Hydrodynamics and Motion Control, John Wiley, Chichester, UK, doi: 10.1002/9781119994138.
• 22. M. Breivik, T. I. Fossen, “Principles of guidance-based path following in 2D and 3D,” in 44th IEEE Conference on Decision Control/European Control Conference (CCDECC), Seville, Spain, 2005.
• 23. T. Liu, Y. L. Hu, and H. Xu, “Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control,” Complexity, vol. 2021, Apr 2021, doi: 10.1155/2021/6649625.
• 24. S. Fujimoto, H. V. Hoof, D. Meger, Addressing Function Approximation Error in Actor-Critic Methods, ICML, 2018.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).