An automated driving strategy generating method based on WGAIL–DDPG

• Autorzy: Zhang Mingheng , Wan Xing , Gang Longhui , Lv Xinfei , Wu Zengwen , Liu Zhaoyang

Identyfikatory

DOI

10.34768/amcs-2021-0031

• Języki publikacji: EN

Abstrakty

EN

Reliability, efficiency and generalization are basic evaluation criteria for a vehicle automated driving system. This paper proposes an automated driving decision-making method based on the Wasserstein generative adversarial imitation learning–deep deterministic policy gradient (WGAIL–DDPG(λ)). Here the exact reward function is designed based on the requirements of a vehicle’s driving performance, i.e., safety, dynamic and ride comfort performance. The model’s training efficiency is improved through the proposed imitation learning strategy, and a gain regulator is designed to smooth the transition from imitation to reinforcement phases. Test results show that the proposed decision-making model can generate actions quickly and accurately according to the surrounding environment. Meanwhile, the imitation learning strategy based on expert experience and the gain regulator can effectively improve the training efficiency for the reinforcement learning model. Additionally, an extended test also proves its good adaptability for different driving conditions.

Słowa kluczowe

EN

automated driving system; deep learning; deep reinforcement learning; imitation learning; deep deterministic policy gradient

PL

system jezdny; uczenie głębokie; uczenie przez naśladowanie

• Wydawca: Oficyna Wydawnicza Uniwersytetu Zielonogórskiego
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2021
• Tom: Vol. 31, no. 3
• Strony: 461--470
• Opis fizyczny: Bibliogr. 17 poz., rys., tab., wykr.

Twórcy

autor

Zhang Mingheng

• State Key Laboratory of Structure Analysis for Industrial Equipment, Dalian University of Technology, Liaoning, Dalian 116024, China; School of Automotive Engineering, Dalian University of Technology, Liaoning, Dalian 116024, China

autor

Wan Xing

• School of Automotive Engineering, Dalian University of Technology, Liaoning, Dalian 116024, China

autor

Gang Longhui

• School of Navigation, Dalian Maritime University, Liaoning, Dalian 116026, China

autor

Lv Xinfei

• School of Automotive Engineering, Dalian University of Technology, Liaoning, Dalian 116024, China

autor

Wu Zengwen

• School of Automotive Engineering, Dalian University of Technology, Liaoning, Dalian 116024, China

autor

Liu Zhaoyang

• School of Automotive Engineering, Dalian University of Technology, Liaoning, Dalian 116024, China

Bibliografia

• [1] Anderson, C.W., Lee, M. and Elliott, D.L. (2015). Faster reinforcement learning after pretraining deep networks to predict state dynamics, 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, pp. 1–7.
• [2] Bai, Z., Shangguan, W., Cai, B. and Chai, L. (2019). Deep reinforcement learning based high-level driving behavior decision-making model in heterogeneous traffic, 2019 Chinese Control Conference (CCC), Guangzhou, China, pp. 8600–8605.
• [3] Chang, B.-J., Hwang, R.-H., Tsai, Y.-L., Yu, B.-H. and Liang, Y.-H. (2019). Cooperative adaptive driving for platooning autonomous self driving based on edge computing, International Journal of Applied Mathematics and Computer Science 29(2): 213–225, DOI: 10.2478/amcs-2019-0016.
• [4] Chen, X., Tian, G., Miao, Y. and Gong, J.-w. (2017). Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic, Transactions of Beijing Institute of Technology 37(5): 491–496.
• [5] Dossa, R.F., Lian, X., Nomoto, H., Matsubara, T. and Uehara, K. (2020). Hybrid of reinforcement and imitation learning for human-like agents, IEICE Transactions on Information and Systems 103(9): 1960–1970.
• [6] Gao, H., Shi, G.,Wang, K., Xie, G. and Liu, Y. (2019). Research on decision-making of autonomous vehicle following based on reinforcement learning method, Industrial Robot: The International Journal of Robotics Research and Application 46(3): 444–452.
• [7] Gu, X., Han, Y. and Yu, J. (2020). Vehicle lane-changing decision model based on decision mechanism and support vector machine, Journal of Harbin Institute of Technology 52(07): 111–121.
• [8] Hedjar, R. and Bounkhel, M. (2019). An automatic collision avoidance algorithm for multiple marine surface vehicles, International Journal of Applied Mathematics and Computer Science 29(4): 759–768, DOI: 10.2478/amcs-2019-0056.
• [9] Ho, J. and Ermon, S. (2016). Generative adversarial imitation learning, Advances in Neural Information Processing Systems 29: 4572–4580.
• [10] Pomerleau, D. (1998). ALVINN: An autonomous land vehicle in a neural network, in D.S. Touretzky (Ed), Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers, Burlington.
• [11] Xia,W. and Li, H. (2017). Training method of automatic driving strategy based on deep reinforcement learning, Journal of Integration Technology 6(3): 29–40.
• [12] Xiong, G.-m., Li, Y. and Wang, S.-y. (2015). A behavior prediction and control method based on FSM for intelligent vehicles in an intersection, Transactions of Beijing Institute of Technology 35(1): 7.
• [13] Xu, H., Gao, Y., Yu, F. and Darrell, T. (2017). End-to-end learning of driving models from large-scale video datasets, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 2174–2182.
• [14] Zhu, M., Wang, X. and Wang, Y. (2018). Human-like autonomous car-following model with deep reinforcement learning, Transportation Research C: Emerging Technologies 97: 348–368.
• [15] Ziegler, J., Bender, P., Schreiber, M., Lategahn, H., Strauss, T., Stiller, C., Dang, T., Franke, U., Appenrodt, N., Keller, C.G., Kaus, E., Herrtwich, R.G., Rabe, C., Pfeiffer, D., Lindner, F., Stein, F., Erbs, F., Enzweiler, M., Knoppel, C., Hipp, J., Haueis, M., Trepte, M., Brenk, C., Tamke, A., Ghanaat, M., Braun, M., Joos, A., Fritz, H., Mock, H., Hein, M. and Zeeb, E. (2014). Making Bertha drive—An autonomous journey on a historic route, IEEE Intelligent Transportation Systems Magazine 6(2): 8–20.
• [16] Zong, X., Xu, G., Yu, G., Su, H. and Hu, C. (2017). Obstacle avoidance for self-driving vehicle with reinforcement learning, SAE International Journal of Passenger Cars—Electronic and Electrical Systems 11(07-11-01-0003): 30–39.
• [17] Zou, Q., Xiong, K. and Hou, Y. (2020). An end-to-end learning of driving strategies based on DDPG and imitation learning, 2020 Chinese Control And Decision Conference (CCDC), Hefei, China, pp. 3190–3195.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).