Online Lifelong Optimal Tracking Control of Uncertain Nonlinear Continuous-time Strict-feedback Systems using Deep Neural Networks

Abstract

A novel integral reinforcement learning (IRL)-based optimal trajectory tracking scheme for nonlinear continuous-time systems in strict feedback form is introduced by using backstepping and multilayer or deep neural networks (DNNs). The proposed method employs a dynamic surface control-based technique in an optimal framework to relax the need for repeatedly computing the derivatives of virtual controllers at each step of the backstepping process. An online singular value decomposition (SVD)-of the activation function gradient-based actor–critic DNN at each step of the backstepping process is employed to minimize a discounted value function. Novel online SVD-based weight update laws, which are shown to mitigate vanishing gradient, for the actor and critic DNNs are derived by using control input error and Bellman error respectively. A new online lifelong learning (LL) technique using Bellman residual and control input errors to overcome the issue of catastrophic forgetting in both critic and actor DNNs is also attempted, and closed-loop stability is analyzed and demonstrated. The effectiveness of the proposed method is shown in simulation on mobile robot tracking and ship autopilot, which demonstrates a 76% total cost reduction when compared to the literature.

Department(s)

Electrical and Computer Engineering

Second Department

Computer Science

Comments

Office of Naval Research, Grant N00014-21-1-2232

Keywords and Phrases

Deep neural networks; Online lifelong learning; Optimal control; Reinforcement learning; Strict-feedback system

International Standard Serial Number (ISSN)

1879-2782; 0893-6080

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2025 Elsevier, All rights reserved.

Publication Date

01 Nov 2025

Link to Full Text

Share

 
COinS