Hamiltonian-Driven Adaptive Dynamic Programming for Continuous Nonlinear Dynamical Systems
This paper presents a Hamiltonian-driven framework of adaptive dynamic programming (ADP) for continuous time nonlinear systems, which consists of evaluation of an admissible control, comparison between two different admissible policies with respect to the corresponding the performance function, and the performance improvement of an admissible control. It is showed that the Hamiltonian can serve as the temporal difference for continuous-time systems. In the Hamiltonian-driven ADP, the critic network is trained to output the value gradient. Then, the inner product between the critic and the system dynamics produces the value derivative. Under some conditions, the minimization of the Hamiltonian functional is equivalent to the value function approximation. An iterative algorithm starting from an arbitrary admissible control is presented for the optimal control approximation with its convergence proof. The implementation is accomplished by a neural network approximation. Two simulation studies demonstrate the effectiveness of Hamiltonian-driven ADP.
Y. Yang et al., "Hamiltonian-Driven Adaptive Dynamic Programming for Continuous Nonlinear Dynamical Systems," IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 8, pp. 1929-1940, Institute of Electrical and Electronics Engineers (IEEE), Aug 2017.
The definitive version is available at https://doi.org/10.1109/TNNLS.2017.2654324
Electrical and Computer Engineering
Intelligent Systems Center
Keywords and Phrases
Adaptive control systems; Approximation algorithms; Continuous time systems; Dynamical systems; Hamiltonians; Iterative methods; Nonlinear control systems; Nonlinear dynamical systems; Adaptive dynamic programming; Continuous time nonlinear systems; Hamiltonian functional; Iterative algorithm; Neural network approximation; Performance functions; Temporal differences; Value function approximation; Dynamic programming; Adaptive dynamic programming (ADP); Convergence proof; Hamiltonian-driven framework; Neural network (NN) approximation; Value function
International Standard Serial Number (ISSN)
Article - Journal
© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
01 Aug 2017