PER-TD3 Integrated with HER Mechanism: Improving Training Efficiency and Control Accuracy for PEMFC Differential Pressure Control

The cathode and anode differential pressure control of a proton exchange membrane fuel cell (PEMFC) directly affects its service life and operating efficiency. Existing control methods find it difficult to cope with strong nonlinear perturbations, and fixed differential pressure control is prone to pressure overshoot and threshold exceedance, resulting in unstable pressure regulation. In order to solve the current research problems, a reinforcement learning method based on hybrid experience replay (HP-TD3) is proposed. A CART-based algorithm is first used to classify the states of the test load, and a load-related segmented reward function is designed. In addition, a hindsight experience replay (HER) mechanism is incorporated into the Priority Experience Replay Twin Delayed Deep Deterministic Policy Gradient (PER-TD3) framework to improve sample utilization efficiency and training stability. Finally, the performance of HP-TD3 and its ability to cope with nonlinear disturbances are verified on a fuel cell control unit hardware-in-the-loop (FCU-HIL) platform. In the A test load (frequent switching and high low-load proportion), the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and the degradation index of the fuel cell dynamic performance (Δfc) of HP-TD3 are respectively reduced by 17.4%, 20.5%, and 13.3% compared to P-TD3; in the B test load (high-load operation and low switching frequency), these indicators are reduced by 25.7%, 29.4%, and 15.4% respectively.