Developing a controller for an adaptive cruise control (ACC) system: utilizing deep reinforcement learning (DRL) approach

• Autorzy: Rizehvandi Ali , Azadi Shahram

Identyfikatory

DOI

https://doi.org/10.20858/sjsutst.2024.125.16

• Języki publikacji: EN

Abstrakty

EN

The addition of Adaptive Cruise Control (ACC) to vehicles enables automatic speed adjustments based on traffic conditions after the driver sets the maximum speed, freeing them to concentrate on steering. This study is dedicated to the development of a passenger car ACC system using Deep Reinforcement Learning (DRL). A critical aspect of this ACC system is its capability to regulate the distance between vehicles by taking into account preceding and following vehicle speeds. It considers three primary inputs: the memory-stored speed of the following vehicle, the lead time specified by the driver, and the radar-measured distance. By adapting speed in different traffic scenarios, the system contributes to averting potential accidents. This research delves into constructing a controller that utilizes the Deep Deterministic Policy Gradient (DDPG) algorithm and compares its outcomes with those from the DQN algorithm. The DDPG controller supervises the longitudinal control actions of a vehicle, enabling it to execute stopping and moving maneuvers safely and efficiently.

Słowa kluczowe

EN

adaptive cruise control system; deep reinforcement learning method; deep deterministic policy gradient algorithm

PL

tempomat adaptacyjny; uczenie głębokie ze wzmocnieniem

• Wydawca: Wydawnictwo Politechniki Śląskiej
• Czasopismo: Zeszyty Naukowe. Transport / Politechnika Śląska
• Rocznik: 2024
• Tom: z. 125
• Strony: 243--257
• Opis fizyczny: Bibliogr. 16 poz.

Twórcy

autor

Rizehvandi Ali

• Department of Mechanical Engineering, K.N. Toosi University of Technology, Tehran 15418-49611, Iran

• https://orcid.org/0009-0007-9259-2101

autor

Azadi Shahram

• Department of Mechanical Engineering, K.N. Toosi University of Technology, Tehran 15418-49611, Iran

• https://orcid.org/0000-0002-2060-6988

Bibliografia

• 1. Li J., H. Zhang, H. Zhang, Y. Yu. 2021. “Deep reinforcement learning for adaptive cruise control in mixed traffic scenarios”. Transportation Research Part C: Emerging Technologies 124: 102908.
• 2. Huang W., Y. Qiu, L. Zhang, Y. Liu. 2022. “Reinforcement learning-based adaptive cruise control with safe exploration strategies”. IEEE Robotics and Automation Letters 7(2): 4562-4569.
• 3. Wang L., Z. Chen, Y. Zheng. 2021. “Improving safety and efficiency in adaptive cruise control through deep reinforcement learning”. Transportation Research Part B: Methodological 148: 193-215.
• 4. Yang T., Y. Liu, Y. Li. 2022. “Deep reinforcement learning for adaptive cruise control under uncertain traffic conditions”. IEEE Access 10: 5678-5689.
• 5. Zhu Q., R. Chen, Z. Wang, L. Zhang. 2021. “Deep reinforcement learning for adaptive cruise control with model-free uncertainty estimation”. IEEE Transactions on Cybernetics 51(5): 2341-2350.
• 6. Park J., J. Lee, S. Kim. 2022. “Enhancing adaptive cruise control using deep reinforcement learning with temporal difference learning”. Expert Systems with Applications 189: 116028.
• 7. Liu T., B. Tian, Y. Ai, L. Chen, F. Liu, D. Cao. 2019. “Dynamic states prediction in autonomous vehicles: Comparison of three different methods”. Proc. IEEE Intell. Transp. Syst. Conf. (ITSC): 3750-3755.
• 8. Sutton R.S., A.G. Barto. 2018. Reinforcement Learning: An Introduction (2nd ed.). Cambridge, MA, USA: MIT Press.
• 9. Li G., L. Yang, S. Li, X. Luo, X. Qu, G. Paul. 2022. “Human-like decision-making of artificial drivers in intelligent transportation systems: An end-to-end driving behavior prediction approach”. IEEE Intelligent Transportation Systems Magazine 14(1): 45-56. DOI: https://doi.org/10.1109/MITS.2022.3085986.
• 10. Zhang Q., J. Lin, Q. Sha, B. He, G. Li. 2020. “Deep interactive reinforcement learning for path following of autonomous underwater vehicle”. IEEE Access 8: 24258-24268.
• 11. Zhang D., N.L. Azad, S. Fischmeister, S. Marksteiner. 2023. “Zeroth-Order Optimization Attacks on Deep Reinforcement Learning-Based Lane Changing Algorithms for Autonomous Vehicles”. Proceedings of the International Conference on Informatics in Control, Automation and Robotics 1: 665-673. DOI: 10.5220/0012187700003543.
• 12. Lv K., X. Pei, C. Chen, J. Xu. 2022. “A safe and efficient lane change decision-making strategy of autonomous driving based on deep reinforcement learning”. Mathematics 10(9): article 1551: 1-14.
• 14. Zhang, Q., J. Lin, Q. Sha, B. He, G. Li. 2020. “Deep interactive reinforcement learning for path following of autonomous underwater vehicle”. IEEE Access 8: 24258-24268.
• 15. Zhang D., N.L. Azad, S. Fischmeister, S. Marksteiner. 2023. “Zeroth-Order Optimization Attacks on Deep Reinforcement Learning-Based Lane Changing Algorithms for Autonomous Vehicles. Proceedings of the International Conference on Informatics in Control, Automation and Robotics 1: 665-673. DOI: 10.5220/0012187700003543.
• 16. Research Gate. Available at: https://www.researchgate.net/figure/The-structure-of-the-DDPGmodel-6_fig5_356879301.