Optimal Control of Unknown Affine Nonlinear Discrete-time Systems using Offline-trained Neural Networks with Proof of Convergence

Author

Travis Dierks
Balaje T. Thumati
S. Jagannathan, Missouri University of Science and TechnologyFollow

Abstract

The optimal control of linear systems accompanied by quadratic cost functions can be achieved by solving the well-known Riccati equation. However, the optimal control of nonlinear discrete-time systems is a much more challenging task that often requires solving the nonlinear Hamilton-Jacobi-Bellman (HJB) equation. in the recent literature, discrete-time approximate dynamic programming (ADP) techniques have been widely used to determine the optimal or near optimal control policies for affine nonlinear discrete-time systems. However, an inherent assumption of ADP requires the value of the controlled system one step ahead and at least partial knowledge of the system dynamics to be known. in this work, the need of the partial knowledge of the nonlinear system dynamics is relaxed in the development of a novel approach to ADP using a two-part process: online system identification and offline optimal control training. First, in the system identification process, a neural network (NN) is tuned online using novel tuning laws to learn the complete plant dynamics so that a local asymptotic stability of the identification error can be shown. Then, using only the learned NN system model, offline ADP is attempted resulting in a novel optimal control law. the proposed scheme does not require explicit knowledge of the system dynamics as only the learned NN model is needed. the proof of convergence is demonstrated. Simulation results verify theoretical conjecture. © 2009 Elsevier Ltd. All rights reserved.

Recommended Citation

T. Dierks et al., "Optimal Control of Unknown Affine Nonlinear Discrete-time Systems using Offline-trained Neural Networks with Proof of Convergence," Neural Networks, vol. 22, no. 5 thru 6, pp. 851 - 860, Elsevier, Jul 2009.

The definitive version is available at https://doi.org/10.1016/j.neunet.2009.06.014

Department(s)

Electrical and Computer Engineering

Second Department

Computer Science

Comments

National Science Foundation, Grant 0621924

Keywords and Phrases

Heuristic dynamic programming; Neural network; Nonlinear optimal control; System identification

International Standard Serial Number (ISSN)

0893-6080

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Elsevier, All rights reserved.

Publication Date

01 Jul 2009

PubMed ID

19596551