Reinforcement learning-based adaptive traffic signal control invariant to traffic signal configuration

Yumaganov A.S., Agafonov A.A., Myasnikov V.V.
 pdf (425K)

In this paper, we propose an adaptive traffic signal control method invariant to the configuration of the traffic signal. The proposed method uses one neural network model to control traffic signals of various configurations, differing both in the number of controlled lanes and in the used traffic light control cycle (set of phases). To describe the state space, both dynamic information about the current state of the traffic flow and static data about the configuration of a controlled intersection are used. To increase the speed of model training and reduce the required amount of data required for model convergence, it is proposed to use an “expert” who provides additional data for model training. As an expert, we propose to use an adaptive control method based on maximizing the weighted flow of vehicles through an intersection. Experimental studies of the effectiveness of the developed method were carried out in a microscopic simulation software package. The obtained results confirmed the effectiveness of the proposed method in different simulation scenarios. The possibility of using the developed method in a simulation scenario that is not used in the training process was shown. We provide a comparison of the proposed method with other baseline solutions, including the method used as an “expert”. In most scenarios, the developed method showed the best results by average travel time and average waiting time criteria. The advantage over the method used as an expert, depending on the scenario under study, ranged from 2% to 12% according to the criterion of average vehicle waiting time and from 1% to 7% according to the criterion of average travel time.

Keywords: traffic signal control, reinforcement learning, connected vehicles, imitation modelling
Citation in English: Yumaganov A.S., Agafonov A.A., Myasnikov V.V. Reinforcement learning-based adaptive traffic signal control invariant to traffic signal configuration // Computer Research and Modeling, 2024, vol. 16, no. 5, pp. 1253-1269
Citation in English: Yumaganov A.S., Agafonov A.A., Myasnikov V.V. Reinforcement learning-based adaptive traffic signal control invariant to traffic signal configuration // Computer Research and Modeling, 2024, vol. 16, no. 5, pp. 1253-1269
DOI: 10.20537/2076-7633-2024-16-5-1253-1269
URL: http://crm-en.ics.org.ru/journal/article/3527/
Creative Commons License This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License.

Indexed in Scopus

Full-text version of the journal is also available on the web site of the scientific electronic library eLIBRARY.RU

The journal is included in the Russian Science Citation Index

The journal is included in the RSCI

International Interdisciplinary Conference "Mathematics. Computing. Education"

Copyright © 2009–2024 Institute of Computer Science