Approximate Dynamic Programming Solutions with a Single Network Adaptive Critic for a Class of Nonlinear Systems

Abstract

Approximate dynamic programming (ADP) formulation implemented with an adaptive critic (AC)-based neural network (NN) structure has evolved as a powerful technique for solving the Hamilton-Jacobi-Bellman (HJB) equations. As interest in ADP and the AC solutions are escalating with time, there is a dire need to consider possible enabling factors for their implementations. A typical AC structure consists of two interacting NNs, which is computationally expensive. In this paper, a new architecture, called the “cost-function-based single network adaptive critic (J-SNAC)” is presented, which eliminates one of the networks in a typical AC structure. This approach is applicable to a wide class of nonlinear systems in engineering. In order to demonstrate the benefits and the control synthesis with the J-SNAC, two problems have been solved with the AC and the J-SNAC approaches. Results are presented, which show savings of about 50% of the computational costs by J-SNAC while having the same accuracy levels of the dual network structure in solving for optimal control. Furthermore, convergence of the J-SNAC iterations, which reduces to a least-squares problem, is discussed; for linear systems, the iterative process is shown to reduce to solving the familiar algebraic Ricatti equation. © 2011 South China University of Technology, Academy of Mathematics and Systems Science, Chinese Academy of Sciences and Springer Verlag Berlin Heidelberg.

Department(s)

Mechanical and Aerospace Engineering

International Standard Serial Number (ISSN)

1672-6340

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2011 Springer Verlag, All rights reserved.

Publication Date

01 Jan 2011

Link to Full Text

Share

 
COinS