Reinforcement Learning for Combinatorial Optimization of Train Timetable Rescheduling

Qi Shi, Xuewu Dai, Dongliang Cui*, Lijuan Cheng

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Downloads (Pure)


With the advancement of computational intelligence, reinforcement learning has been proposed as a solution for train timetable rescheduling (TTR). However, traditional Q-learning approaches face challenges of large memory overhead and slow convergence rate due to the large state and action spaces in TTR. Moreover, Q-learning suffers from limited generalization ability, requiring retraining of the Q-table for handling new delay scenarios. To accelerate convergence speed and improve generalization ability, this paper proposes a deep reinforcement learning (DQN) approach for TTR. An adaptive action generation method is employed to address the slow convergence issue caused by a large action search space. In the proposed DQN, a target network and an experience replay mechanism are used to enhance the generalization ability of reinforcement learning. The performance of the proposed DQN is evaluated through simulation on a high-speed railway in China. Compared to the First-Come, First-Served (FCFS) baseline method, the validated algorithm yields an 80 % improvement in the generated schedules over FCFS. Moreover, the online decision-making process of the tested algorithm takes only 1-2 seconds, outperforming FCFS in terms of efficiency and effectiveness.

Original languageEnglish
Title of host publicationProceedings - 2023 IEEE SmartWorld, Ubiquitous Intelligence and Computing, Autonomous and Trusted Vehicles, Scalable Computing and Communications, Digital Twin, Privacy Computing and Data Security, Metaverse, SmartWorld/UIC/ATC/ScalCom/DigitalTwin/PCDS/Metaverse 2023
Place of PublicationPiscataway, NY
Number of pages7
ISBN (Electronic)9798350319804
ISBN (Print)9798350319811
Publication statusPublished - 28 Aug 2023
Event9th IEEE Smart World Congress, SWC 2023 - Portsmouth, United Kingdom
Duration: 28 Aug 202331 Aug 2023


Conference9th IEEE Smart World Congress, SWC 2023
Country/TerritoryUnited Kingdom

Cite this