This study provides a lifelong integral reinforcement learning (LIRL)-based optimal tracking scheme for uncertain nonlinear continuous-time (CT) systems using multilayer neural network (MNN). In this LIRL framework, the optimal control policies are generated by using both the critic neural network (NN) weights and single-layer NN identifier. The critic MNN weight tuning is accomplished using an improved singular value decomposition (SVD) of its activation function gradient. The NN identifier, on the other hand, provides the control coefficient matrix for computing the control policies. An online weight velocity attenuation (WVA)-based consolidation scheme is proposed wherein the significance of weights is derived by using Hamilton-Jacobi-Bellman (HJB) error. This WVA term is incorporated in the critic MNN update law to overcome catastrophic forgetting. Lyapunov stability is employed to demonstrate the uniform ultimate boundedness of the overall closed-loop system. Finally, a numerical example of a two-link robotic manipulator supports the theoretical claims.
I. Ganie and S. Jagannathan, "Continual Optimal Adaptive Tracking Of Uncertain Nonlinear Continuous-time Systems Using Multilayer Neural Networks," Proceedings of the American Control Conference, pp. 3395 - 3400, Institute of Electrical and Electronics Engineers, Jan 2023.
The definitive version is available at https://doi.org/10.23919/ACC55779.2023.10156466
Electrical and Computer Engineering
Keywords and Phrases
Catastrophic forgetting; Continual learning; Lifelong learning; Multilayer neural networks; Reinforcement learning
International Standard Serial Number (ISSN)
Article - Conference proceedings
© 2023 Institute of Electrical and Electronics Engineers, All rights reserved.
01 Jan 2023