Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems
Abstract
The stochastic optimal control of nonlinear networked control systems (NNCSs) using neuro-dynamic programming (NDP) over a finite time horizon is a challenging problem due to terminal constraints, system uncertainties, and unknown network imperfections, such as network-induced delays and packet losses. Since the traditional iteration or time-based infinite horizon NDP schemes are unsuitable for NNCS with terminal constraints, a novel time-based NDP scheme is developed to solve finite horizon optimal control of NNCS by mitigating the above-mentioned challenges. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix that is subsequently utilized in conjunction with the critic and actor NNs to determine a time-based stochastic optimal control input over finite horizon in a forward-in-time and online manner. Eventually, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to optimal value within finite time. The simulation results are included to show the effectiveness of the proposed scheme.
Recommended Citation
H. Xu and J. Sarangapani, "Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems," IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 3, pp. 472 - 485, Institute of Electrical and Electronics Engineers (IEEE), Mar 2015.
The definitive version is available at https://doi.org/10.1109/TNNLS.2014.2315622
Department(s)
Electrical and Computer Engineering
Keywords and Phrases
Control systems; Control theory; Dynamic programming; Optimal control systems; Social networking (online); Stochastic control systems; Stochastic systems; Finite horizon optimal control; Network-induced delays; Neuro-Dynamic Programming; Nonlinear networked control systems; Nonlinear networked control systems (NNCS); On-line neural networks; Stochastic optimal control; Uniformly ultimately bounded; Networked control systems; Artificial neural network; Nonlinear system; Statistics; Time; Neural Networks (Computer); Nonlinear Dynamics; Stochastic Processes; Time Factors; Neuro-dynamic programming (NDP); Nonlinear networked control system (NNCS); Stochastic optimal control
International Standard Serial Number (ISSN)
2162-237X
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Mar 2015
Comments
This work was supported by the National Science Foundation under Grant ECCS 1128281.