Optimal Regulation of Uncertain Dynamic Systems using Adaptive Dynamic Programming

Abstract

In this tutorial paper, the finite-horizon optimal adaptive regulation of linear and nonlinear dynamic systems with unknown system dynamics is presented in a forward-in-time manner using adaptive dynamic programming (ADP). An adaptive estimator (AE) is introduced with the idea of Q-learning to relax the requirement of system dynamics in the case of linear system, while neural network-based identifier is utilised for nonlinear systems. The time-varying nature of the solution to the Bellman/Hamilton–Jacobi–Bellman equation is handled by utilising a time-dependent basis function, while the terminal constraint is incorporated as part of the update law of the AE/Identifier in solving the optimal feedback control. Utilising an initial admissible control, the proposed optimal regulation scheme of the uncertain linear and nonlinear system yields a forward-in-time and online solution without using value and/or policy iterations. An adaptive observer is utilised for linear systems in order to relax the need for state availability so that the optimal adaptive control design depends only on the reconstructed states. Finally, the optimal control is covered for nonlinear-networked control systems where in the feedback loop is closed via a communication network. Effectiveness of the proposed approach is verified by simulation results. The end result is a variant of a roll-out scheme in ADP wherein an initial admissible policy is selected as the base policy and the control policy is enhanced using a one-time policy improvement at each sampling interval.

Department(s)

Electrical and Computer Engineering

Second Department

Computer Science

Keywords and Phrases

adaptive dynamic programming; finite horizon; optimal control

International Standard Serial Number (ISSN)

2330-7714; 2330-7706

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Taylor and Francis Group; Taylor and Francis, All rights reserved.

Publication Date

03 Jul 2014

Link to Full Text

Share

 
COinS