Abstract
This study provides a lifelong integral reinforcement learning (LIRL)-based optimal tracking scheme for uncertain nonlinear continuous-time (CT) systems using multilayer neural network (MNN). In this LIRL framework, the optimal control policies are generated by using both the critic neural network (NN) weights and single-layer NN identifier. The critic MNN weight tuning is accomplished using an improved singular value decomposition (SVD) of its activation function gradient. The NN identifier, on the other hand, provides the control coefficient matrix for computing the control policies. An online weight velocity attenuation (WVA)-based consolidation scheme is proposed wherein the significance of weights is derived by using Hamilton-Jacobi-Bellman (HJB) error. This WVA term is incorporated in the critic MNN update law to overcome catastrophic forgetting. Lyapunov stability is employed to demonstrate the uniform ultimate boundedness of the overall closed-loop system. Finally, a numerical example of a two-link robotic manipulator supports the theoretical claims.
Recommended Citation
I. Ganie and S. Jagannathan, "Continual Optimal Adaptive Tracking Of Uncertain Nonlinear Continuous-time Systems Using Multilayer Neural Networks," Proceedings of the American Control Conference, pp. 3395 - 3400, Institute of Electrical and Electronics Engineers, Jan 2023.
The definitive version is available at https://doi.org/10.23919/ACC55779.2023.10156466
Department(s)
Electrical and Computer Engineering
Second Department
Computer Science
Keywords and Phrases
Catastrophic forgetting; Continual learning; Lifelong learning; Multilayer neural networks; Reinforcement learning
International Standard Serial Number (ISSN)
0743-1619
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2023 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
01 Jan 2023
Comments
Office of Naval Research, Grant N00014-21-1-2232