Improving the quality of route generation in SUMO based on data from detectors using reinforcement learning

Salenek I.A., Seliverstov Y.A., Seliverstov S.A., Sofronova E.A.
 pdf (240K)

This work provides a new approach for constructing high-precision routes based on data from transport detectors inside the SUMO traffic modeling package. Existing tools such as flowrouter and routeSampler have a number of disadvantages, such as the lack of interaction with the network in the process of building routes. Our rlRouter uses multi-agent reinforcement learning (MARL), where the agents are incoming lanes and the environment is the road network. By performing actions to launch vehicles, agents receive a reward for matching data from transport detectors. Parameter Sharing DQN with the LSTM backbone of the Q-function was used as an algorithm for multi-agent reinforcement learning.

Since the rlRouter is trained inside the SUMO simulation, it can restore routes better by taking into account the interaction of vehicles within the network with each other and with the network infrastructure. We have modeled diverse traffic situations on three different junctions in order to compare the performance of SUMO’s routers with the rlRouter. We used Mean Absoluter Error (MAE) as the measure of the deviation from both cumulative detectors and routes data. The rlRouter achieved the highest compliance with the data from the detectors. We also found that by maximizing the reward for matching detectors, the resulting routes also get closer to the real ones. Despite the fact that the routes recovered using rlRouter are superior to the routes obtained using SUMO tools, they do not fully correspond to the real ones, due to the natural limitations of induction-loop detectors. To achieve more plausible routes, it is necessary to equip junctions with other types of transport counters, for example, camera detectors.

Keywords: transport modeling, multi-agent reinforcement learning, intelligent transport systems
Citation in English: Salenek I.A., Seliverstov Y.A., Seliverstov S.A., Sofronova E.A. Improving the quality of route generation in SUMO based on data from detectors using reinforcement learning // Computer Research and Modeling, 2024, vol. 16, no. 1, pp. 137-146
Citation in English: Salenek I.A., Seliverstov Y.A., Seliverstov S.A., Sofronova E.A. Improving the quality of route generation in SUMO based on data from detectors using reinforcement learning // Computer Research and Modeling, 2024, vol. 16, no. 1, pp. 137-146
DOI: 10.20537/2076-7633-2024-16-1-137-146
URL: http://crm-en.ics.org.ru/journal/article/3439/
Creative Commons License This work is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License.

Indexed in Scopus

Full-text version of the journal is also available on the web site of the scientific electronic library eLIBRARY.RU

The journal is included in the Russian Science Citation Index

The journal is included in the RSCI

International Interdisciplinary Conference "Mathematics. Computing. Education"

Copyright © 2009–2024 Institute of Computer Science