Reinforcement Learning for Combinatorial Optimization of Train Timetable Rescheduling

With the advancement of computational intelligence, reinforcement learning has been proposed as a solution for train timetable rescheduling (TTR). However, traditional Q-learning approaches face challenges of large memory overhead and slow convergence rate due to the large state and action spaces in TTR. Moreover, Q-learning suffers from limited generalization ability, requiring retraining of the Q-table for handling new delay scenarios. To accelerate convergence speed and improve generalization ability, this paper proposes a deep reinforcement learning (DQN) approach for TTR. An adaptive action generation method is employed to address the slow convergence issue caused by a large action search space. In the proposed DQN, a target network and an experience replay mechanism are used to enhance the generalization ability of reinforcement learning. The performance of the proposed DQN is evaluated through simulation on a high-speed railway in China. Compared to the First-Come, First-Served (FCFS) baseline method, the validated algorithm yields an 80 % improvement in the generated schedules over FCFS. Moreover, the online decision-making process of the tested algorithm takes only 1-2 seconds, outperforming FCFS in terms of efficiency and effectiveness.