Safe Reinforcement Learning for Power Allocation of Hybrid Energy Storage Systems in Electric Vehicles using PPO and Predictive Safety Filter

Xuetong Xu, Lijun Zhang*, Jingang Lai, Longzhi Yang, Jiangwen Xiao

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Energy management of lithium-ion batteries to extend their lifespan while considering their heat generation is pivotal for their cost-effective and safe operation. For this purpose, we present a power allocation strategy for battery-supercapacitor hybrid energy storage systems used in electric vehicles. The proposed method combines the advantages of reinforcement learning and a predictive safety filter to devise a safe reinforcement learning solution. What’s more, a tailored incentive reward is designed to guide the training processes of the reinforcement learning taking into account the impacts of battery heating. Comparisons with low pass filtering (LPF), model predictive control (MPC) and twin delayed deep deterministic policy gradient (TD3) algorithms reported in the literature under various operating conditions show the superiority of the proposed approach. In particular, the proposed strategy demonstrates a remarkable reduction in the generalised operating cost, outperforming existing LPF techniques, the MPC method and the TD3 algorithm by 8.0%-30.6%, 0.3%-2.5% and 0.3%-9.4%, respectively. Compared to MPC, the proposed method improves computational efficiency by more than double while ensuring safety. Additionally, it significantly reduces the algorithm training time by almost 97% compared to existing proximal policy optimisation methods. The proposed power allocation strategy can be applied to any system employing a hybrid battery energy storage system to alleviate battery ageing while ensuring operation safety.
Original languageEnglish
Pages (from-to)4024-4034
Number of pages11
JournalIEEE Transactions on Transportation Electrification
Volume11
Issue number1
Early online date29 Aug 2024
DOIs
Publication statusPublished - 3 Feb 2025

Keywords

  • Batteries
  • safety
  • integrated circuit modeling
  • resource management
  • supercapacitors
  • state of charge
  • resistance heating
  • lithium-ion battery
  • electric vehicle
  • hybrid energy storage system
  • power split
  • deep reinforcement learning
  • predictive safety filter
  • Lithium-ion battery

Cite this