Safe Reinforcement Learning for Power Allocation of Hybrid Energy Storage Systems in Electric Vehicles using PPO and Predictive Safety Filter

Energy management of lithium-ion batteries to extend their lifespan while considering their heat generation is pivotal for their cost-effective and safe operation. For this purpose, we present a power allocation strategy for battery-supercapacitor hybrid energy storage systems used in electric vehicles. The proposed method combines the advantages of reinforcement learning and a predictive safety filter to devise a safe reinforcement learning solution. What’s more, a tailored incentive reward is designed to guide the training processes of the reinforcement learning taking into account the impacts of battery heating. Comparisons with low pass filtering (LPF), model predictive control (MPC) and twin delayed deep deterministic policy gradient (TD3) algorithms reported in the literature under various operating conditions show the superiority of the proposed approach. In particular, the proposed strategy demonstrates a remarkable reduction in the generalised operating cost, outperforming existing LPF techniques, the MPC method and the TD3 algorithm by 8.0%-30.6%, 0.3%-2.5% and 0.3%-9.4%, respectively. Compared to MPC, the proposed method improves computational efficiency by more than double while ensuring safety. Additionally, it significantly reduces the algorithm training time by almost 97% compared to existing proximal policy optimisation methods. The proposed power allocation strategy can be applied to any system employing a hybrid battery energy storage system to alleviate battery ageing while ensuring operation safety.