This article addresses a multilayer neural network (MNN)-based optimal adaptive tracking of partially uncertain nonlinear discrete-time (DT) systems in affine form. By employing an actor–critic neural network (NN) to approximate the value function and optimal control policy, the critic NN is updated via a novel hybrid learning scheme, where its weights are adjusted once at a sampling instant and also in a finite iterative manner within the instants to enhance the convergence rate. Moreover, to deal with the persistency of excitation (PE) condition, a replay buffer is incorporated into the critic update law through concurrent learning. To address the vanishing gradient issue, the actor and critic MNN weights are tuned using control input and temporal difference errors (TDEs), respectively. In addition, a weight consolidation scheme is incorporated into the critic MNN update law to attain lifelong learning and overcome catastrophic forgetting, thus lowering the cumulative cost. The tracking error, and the actor and critic weight estimation errors are shown to be bounded using the Lyapunov analysis. Simulation results using the proposed approach on a two-link robot manipulator show a significant reduction in tracking error by $44\%$ and cumulative cost by $31\%$ in a multitask environment.
B. Farzanegan et al., "Optimal Adaptive Tracking Control Of Partially Uncertain Nonlinear Discrete-Time Systems Using Lifelong Hybrid Learning," IEEE Transactions on Neural Networks and Learning Systems, Institute of Electrical and Electronics Engineers, Jan 2023.
The definitive version is available at https://doi.org/10.1109/TNNLS.2023.3301383
Electrical and Computer Engineering
Keywords and Phrases
Artificial neural networks; Convergence; Discrete-time (DT) concurrent learning; experience replay; Hybrid learning; hybrid learning; Iterative methods; lifelong learning (LL); multilayer neural networks (MNNs); Optimal control; optimal tracking control (OTC); Task analysis; Trajectory
International Standard Serial Number (ISSN)
Article - Journal
© 2023 Institute of Electrical and Electronics Engineers, All rights reserved.
01 Jan 2023