Neighboring extremal optimal control theory for parameter-dependent closed-loop laws

Abstract

This study introduces an approach to obtain a neighboring extremal optimal control (NEOC) solution for a closed-loop optimal control problem, applicable to a wide array of nonlinear systems represented by differential equations with affine input coupling, and performance indices that are not necessarily quadratic. The approach involves investigating the variation incurred in the functional form of a known closed-loop optimal control law due to small, known parameter variations in the system equations or the performance index. The NEOC solution can formally be obtained by solving a linear partial differential equation, akin to those encountered in the iterative solution of a nonlinear Hamilton- Jacobi-Bellman (HJB) equation. Just as Galerkin numerical procedures can be used for solving these latter nonlinear equations, we propose a Galerkin-style algorithm for solving the associated NEOC linear partial differential equation, leveraging the use of basis functions that might have been used to solve the underlying HJB equation of the original optimal control problem. The proposed Galerkinbased approach simplifies the NEOC problem by reducing it to the solution of a simple set of linear equations, thereby eliminating the need for a full re-solution of the adjusted optimal control problem. The variation to the optimal performance index is obtained as a function of both the system state and small changes in parameters, allowing the determination of the adjustment to an optimal control law given a small adjustment of parameters in the system or the performance index. Furthermore, we demonstrate the application of the NEOC approach to a simple LQR case, which offers a computationally efficient alternative to directly re-solving the Riccati equation. Moreover, in order to handle large known parameter perturbations, we propose a forward Euler-method that breaks down the single calculation of NEOC into a finite set of multiple steps, each associated with a small parameter variation. Finally, the validity of the claims and theory is supported by theoretical analysis and numerical simulations. (c) 2025 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.