Reinforcement Learning‐Based Sliding Variable Optimal Control for Canonical Nonlinear Systems With Unknown Control Direction

ABSTRACT This paper presents a sliding variable‐based optimal control (SVBOC) framework for canonical nonlinear systems with unknown control direction. The approach integrates reinforcement learning (RL) with neural network (NN) approximation to address the challenge of solving the Hamilton–Jacobi–Bellman (HJB) equation in systems with unknown dynamics. By constructing a sliding variable from tracking errors, the method avoids recursive backstepping steps. Based on the designed sliding variable, a simplified identifier–critic–actor RL structure is proposed without requiring persistent excitation, where the adaptive laws are derived via gradient descent on a positive definite function, and the resulting continuous controller effectively suppresses the chattering phenomenon. Lyapunov analysis guarantees that all closed‐loop signals remain semi‐globally uniformly ultimately bounded (SGUUB), and the tracking error converges near zero. The effectiveness of the approach is demonstrated through two simulation studies.